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SENSITIVITY REDUCTION THROUGH REOPTIMIZATION 

Abstract 

The ability to maintain an optimal solution independent 
of parameter variations is philosophically appealing. 
Unfortunately, whenever the internal system parameters 
change in value, the system usually no longer operates in 
an optimal fashion. Therefore, the purpose, of this research 
is to devise^ a scheme which will continually adjust its 
control strategy in such a manner that the control remains 
optimum for a set of parameter variations. To accomplish 
this, the- Maximum Principle is applied to a truncated Taylor 
series representation of the Hamiltonian of the system with 
parameter variations. An adaptive control strategy is" 
thereby derived. However, the adaptive structure requires 
plant identification, so special attention is given to this 
problem. 
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CHAPTER I 


' INTRODUCTION 

1 . 1 Historical Background 

Progress in the sensitivity problem has not in 
general kept pace with the general advances achieved 
within the framework of optimal control.. Briefly, the 
optimization problem involves transferring the state of 
a given system from some initial state to a given target 

I 

set under various constraints, in such a manner as to 
minimize a given cost functional. The need for a sensi- 
tivity analysis results from the structure of the system's 
dynamical model x(t) = f (x (t) ,u ( t) , a) , where x(t) is the 
state of the system,, u(t) the control effort, and a the 
parameter vector of the system. This model represents 
an ideal plant where the parameters are assumed to be 
known exactly. Such parameters shall be called the 
"nominal parameters." However, the plant parameters may 
change in value during their life span, or even if fixed, 
their precise values may not be known. Therefore, the 
dependence of the state dynamical model upon parameter 
values gives rise to a performance functional, state, and 
optimal control strategy which are dependent upon those 
parameter values. 

Considerable progress has been made in the design of 
linear autonomous systems with controlled sensitivity to 
parameter uncertainty. The foundation work for this class 
of problems is due to Bode 1 , with applications and exten- 
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2 3 

sions by Horowitz ' . Some methods have been presented 

in the field of optimal control systems which are subject 

to parameter variations. There are essentially two 

mutually exclusive philosophies being pursued in this 

4 

field. They are : 

The study of 

(1) Performance Sensitivity 

1 

Sensitivity reduction by considering the 
dependence of the cost functional on the 
parameters.. 

(2) Trajectory Sensitivity 

Sensitivity reduction by considering the 
dependence of the states on the parameters. 

1 . 2 Performance Sensitivity 

Consider any optimal control law which may be im- 
plemented-in either an open-loop or closed-loop structure. 
The question of which structure offers the smallest varia- 
tions in cost for given parameter variations was posed by 
5 

Dorato . From the study of classical control systems, 
the notion of sensitivity reduction to parameter varia- 
tions with the implementation of particular feedback laws 
was developed. The sensitivity index considered was 
usually equivalent to 


oT 3T / sa 


/ 6 a 

' a 


- - , (i-i) 

»0L T 

which is a ratio of the change in transfer function T per 
change in parameter value a. For the open-loop configura- 
tion Sq^ - 2. 
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For certain feedback mechanizations S can be made less 

a 

£ 

than unity implying a reduction in sensitivity. Pagurek 
investigated an analogue of the classical sensitivity 
problem. He considered the cost functional 


Cc(jL,cl) = J L (A(-t),0Ut),a>-dt- / ci- 2 ) 

f. 

Let 6C(u,a) represent the first variation of the cost 
functional. If the plant's initial conditions are known 
and the target set is the whole s-tate space, then Pagurek 


stated that 


8 C ( u.,a ) -SOujCu * Q 


(1-3) 


where the subscripts o and c denote open- and closed- loop 

quantities-. That is, the cost index sensitivity to 

« / ’ 

parameter change is the same for both open- and closed- 

loop transformations provided, of Course, the parameter 

variations are infinitesimally ;small .- The question of how 

much the cost index changes from the nominally optimal : 

cost was left unresolved. Although Pagurek's results 

were for a' special case,- a more general result, according 

to 'Sorbal, was given by Kokotoni'c and Sannuit^. It' was 

found that (1-3) need -not be .zero in general. 

Another performance sensitivity idea considered a 

• ' 8 

game theoretic approach*":" 'Rohrer -and Sobral defined the 
"relative sensitivity index" for a control u(t) to be 



(1-4) 


s 


R 

cuit),co = 


Ccuct),a)-Ccu°a\a3 

| Ccufct^ao I 


where C(u(t) ,a) is defined by (1-2) and u° (t) 
control for the plant having parameters a. 

C cu°ti^a>= rrnn|C(:Li(-t) J a)| 

The "plant' sensitivity" is chosen to be a quantitative 
index of the deviations of C(u(t) ,a) from C(u°(t),a) and 
is defined to be 

S f> cu.t-t» = 7t, g^| S R cu.et^coj (i-6) 

1c 

The optimal design criterion becomes u(t) = u. (t) , where 

So tu *■<*»« min < 

r u t , 

i • D 

ss min m<sx S ( 
a a ( 

Essentially what is being accomplished is u(t) is chosen 
to make C(u(t)) as close as possible to the optimal 
value of C(u(t)) at all values of a. 

1 . 3 Trajectory Sensitivity 

In this method there is generated a trajectory in the 
solution space which is "least" sensitive to parameter 
variations. This is accomplished by constructing an aug- 
mented cost index. Instead of the cost functional being a 
function of x(t), u(t), t, the augmented cost functional 
is a function of x(t) , u(t) , t, and a term which relates 
the change in plant trajectories to parameter variations. 
For example, the cost index which is a function of 
x(t), u (t) , t, and Dx(t)|3cc. A trajectory from the 



is the optimal 
That is, 

(1-5) 



5 


solution space which is least sensitive to parameter 
variations will not in general produce a trajectory which 
is optimal with respect to the original unaugmented cost 

criterion. Trajectory sensitivity methods have been 

9 10 

investigated by Kahne , D'Angelo, Moe , and Hendricks , 

\ 

Bradt"^, and others. 

As the two sensitivity classes differ philosophically, 
so do their applications. Performance, sensitivity methods 
maintain the identity of the original cost functional and 
therefore the results achieved relate to some optimal 
solution. Trajectory sensitivity concerns itself with 
minimizing, in a sense, a cone of trajectories about a 
trajectory which is optimal with respect to an augmented 
cost criterion. 

The last technique described in the performance sen- 
sitivity section is closely allied to an idea proposed by 

* 

Kokotovic and Heller . They have adopted, an approach 
which preserves the concept of optimality in the sense 
that the control law minimizes some given unaugmented cost 
index 

COO = / LCXtt:^ UC-fc),t) dt. (1-8) 

J t 0 

The cost index defined by (1-8) is aesthetically pleasing 
in that most physically meaningful optimal control problems 
have a cost index of this form. Their objective was to 
develop a system which attempts to be optimal for "small" 

k 

Notes, 1967 



6 


parameter variations. Their approach postulated the 
a-priori control law 

* Su - C, Sx +-C 2 Act 

where and C 2 are obtained via the Maximum (Minimum) 
Principle. That is, the control Su, which was called the 
"optimally sensitive control" will tend' to minimize (1-8) 
under the influence of small parameter variations. Be- 
sides the loss in generality due to requiring that the 
feedback control have the structure of (*) , the authors 
left several important facets of the problem unanswered. 
They are : 

(1) How "large" may the allowable parameter 
variations be? 

(2) Is the system's cost of operation (i.e., (1-8)) 
less sensitive to parameter variations than- 

its non-adaptive counterpart (i.e., using 

the fixed nominally optimal control only)? 

(3) Under what conditions will the adaptive 
structure res.ul-t in a cost which is equal 
to, or close to, the true minimal cost 

of operation for a system subject to parameter 
variations? 

Also, the authors' implementation restricted the number of 
parameters considered to be equal to the number of states 
considered. In such cases (see Figure 1.1) 

Ad « -f cSx^. 

However, the authors state that this dimensionality 
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restriction may be overcome. 



Figure 1-1 
Adaptive Controller 
1 . 4 Objectives and Methods 

The objective of this investigation is to develop a 
design method which will operate optimally' (or if not, 
arbitrarily close to optimal) over some allowable set of 
parameter variations. Various types of parameter varia- 
tions will be investigated and given a unified analysis. 
The cost functional considered will be that cost func- 
tional associated with the nominal problem. 

The problem will be formulated in a general mathe- 
matical sense with all structural forms (i.e., constraints 
and control strategy) being a derived result instead of 
being assumed a priori . Arguments pertaining to known 
optimal solutions and their neighboring solutions will be 
used to develop a general Hamiltonian system of equations 
valid over a set of admissible parameters. The desired 
control law will be obtained from a set of necessary 
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oondi.Li.ons pin cod on the general Hamiltonian system. This 
control ] aw will be found to minimize (1-8) over the set 
of well-defined admissible parameter variations. 

Parameter estimation will be found necessary to 
mechanize the derived system. Therefore, questions re- 
lative to parameter estimation and dimensional restric- 
tions will be explored. Various computational devices 
will be developed to satisfy the parameter estimation 

i 

i 

condition . 

Error analysis techniques will be employed, to estab- 
lish bounds on the allowable parameter variations and 
cost index variations. Finally, it will be shown that if 
certain conditions are satisfied, the derived system is 
less sensitive to parameter variations (in the performance 
sensitivity sense) than ,its nominally optimal controlled 
counterpart. 

1 .5 Notation and Symbols 

The following symbols will be used throughout the 
study : 

x state vector 

a parameter vector 

a 11 nominal parameter vector 

k 

a actual parameter vector 

0 

a estimated parameter vector 

or parameter, real 

p costate vector 

u control vector 
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y . 

H ( • ) 
C (u) 


augmented state vector 
Hamiltonian function 

cost index (cost function or performance 
index) 

dz 

dt 



Euclidian q dimensional space’ 

C 1 ] I | The class of all functions with i continuous 
partial derivatives with - 1 respect to all 
arguments on the real interval tel 

P|l| . The class of all piecewise continuous' 
functions 

jj || Norm 

< , > non-degenerate inner product 

Transpose 

o(c) "order" of e 

i 

$v]J Largest integer smaller than or. equal to v 

Variable identification (superscripts) 

* optimal variable 

e adaptive (approximate optimum) variable 

n nominal variable 


actual system variable 
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CHAPTER II 

FORMULATION OF THE PROBLEM 


2 . 1 Necessary Conditions for Optimization 


Consider the autonomous plant 

xtt) - f.<x It), out), CL h t € [to-jT ] 
where: x(t) E n , x(t Q .) = x q (2-1) 


u(t) - 



u. (t) is an admissible measurable 
control functiorj on t£[t Q ,Tj, 


1 

4J 

u 


i = 1 , . • - , r | 


a, the parameter vector will be formally defined as: 


a= 


a, ' 

L Q-m _ 


E 


m 


and ^ a i^ a i an a< ^ m i ss; *-kle measurable parameter of 

the system defined in (2-1), i = 1, , . . , m} . The per- 
formance index C (u) is defined to be: 


C(U)--^<*IT),TXIT»* j f L0!(i),uit))d.t (2-2) 

where <x(T), T x (T)> is the terminal cost functional and 
T is a diagonal positive definite matrix (notice it is 
assumed to be quadratic) • 

Necessary conditions for finding a u(t)eQ, which 
■transfers the state of system defined by (2-1) from x q to 
some target set S at the terminal time T, T given, with: 

fecjt o ,Tj, f: E n X0X CX. -KE n , Le C ^t Q , t| , L: E n X(HE n 


Measurable functions will be considered to be bounded. 
This does not allow for the existence of the Dirac 
delta distributions because distributions are not 
functions . 
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such that C(u) is minimized, subject to the differential 
side constraint (2-1) , are well known from Maximum 
(Minimum) Principle - The Maximum Principle states that 
for a u*(t) to be an optimal control, in that min C(u) = 
C(u*), is that there exists a nontrivial p* (t) such that 
for 

H fX,p,U5CL) ^ L <X 5 tO+<p,f (2-3) 

I 

and tc[t ,t], p(t) cE n t 

(i) p* (t) , X*(t), and u* (t) are solutions of the canonical 


equations 

X*6t) = 



H c x*ct) , p*t).> u*W) 
P 

H(X *co, 

.x 


(t 0 )-x 0 


(2-4) 

(2-5) 


(i±) uln H P*^)' u(t)) = H(x*(t), p*(t), u*(t))(2-6) 

(Iii), and p(t) satisfies the usual transversality condi- 
tions dictated by the target set S. 

Conditions i, ii , and iii are only necessary and not 
sufficient. For example, there exists cases where i, ii, 
and iii are satisfied but the satisfying u(t) is not 
optimal but a singular control belonging to fiis . However, 
for the purposes of this investigation it shall be assumed 
that a unique nonsingular optimal control always exists 
and that it satisfies the Maximum Principle. 

To establish a more definite understanding about the 
character of the plant considered in (2-1) , and therefore 
the problem, the set of admissible parameters must be 


For a listing of transversality conditions for the defined 
problem see pages 306-307 of 12. 
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■"■xp i ojv'fi mo i*'? fully. Th<' n < * L of ndmiss 1 b’l r pn r.iitio I o r:f 
should , 1 'Poop L ,j wi.de spocti.um ol inLi-scpro rations - by 
this it is meant that the admissible parameters should be 
applicable to all systems where the parameters are con- 
sidered to be, for example, initial conditions, or plant 
coefficients, or perhaps a combination of the two. For- 
tunately, there exists a property of ordinary differential 

equations which will establish an equivalency between 

i 

these three parameter cases. 

2.2 Set of Admissible Parameters 

Consider the continuity aspects of a system of first 
order ordinary differential equations for the following 
three cases: 

For xcE n 

(i) x(t) = f (t,x(t) ) , x(t ) = y 

y considered to be a parameter vector, ysE n . 

(ii) x(t) = f ( t ,x , a ) , x (t Q ) = x q fixed 

. a = (a^, . . , , a ) to be considered as a parameter 
vector, cxeE™ 

(iii) Combinations of (i) and (ii) 

. 13 

It xs known that under the proper reduction routine 
( i ) (ii)<£=^ (iii) 

Moreover, these proper reductions preserve all the con- 
tinuity properties of the original system considered. 

Proof : 

(i) =>(ii) 

- y, x (t) a solution of (i) . Then z = & for y 


Let x 


z 
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constant. Therefore, z = x = f (t,z(t) - y) = g(t,z,y) and 
z(t Q ) = 0. Continuity and the smoothness of (i) are pre- 
served under this linear transformation. 

( ii ) -- ■ (i)'. Let z — ^ x ^ , ..., x ^ * ■ * 1 a m ) 

and 

2(t o ) = (j^O, <n O,- « 1 , « m )' 

define i = 9 (t ' z) - ■ . 

Again smoothness is preserved; i.e., if feC^j't , T’J then 
geC q [t 0 ,T]. 

The other implications follow: ^ 

Because there exists an equivalence between' (i) , (ii) , 
and (iii) , a problem formulated in any of the three classes 
may be reduced to any chosen class. Therefore, this in- 
vestigation will only consider problems posed in class (ii) . 

For the sake of completeness, t may also be con- 
sidered to be a parameter in (i) , and therefore in class 
(ii) or (iii) , by increasing the dimension of (i) by one. 
Proof : 

til 

First convert the nonautonomous n order ordinary 


differential system of equations to a (n+1) order ordinary 
autonomous system of differential equations by letting 


X (t) = 1, X , , (t ) - t . 

n+1 n+1 o o 


Define: z(t) = (x- . (t) , X (t) , ..., X (t) )" 

n+1 i n 

Define: g(t,x> 5= (l,f(t,x))^ 


■^By virtue of this transformation only autonomous systems 
need be considered, realizing that a nonautonomous system 
can be reduced to an autonomous equivalent. 
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Then z { L '0 ) - (0^,(0), ^ n (0)) 

J-.(f) - (J^x^d:), x n (-r))" = g(z( / C)) 

2 (r= t Q ) = U n+ 1 (-r= t Q ), x(-c = t o )) = (t D ,x o ) 

Thus, the continuity properties of a system of ordinary 
differential equations with respect to parameters admits 
a specialized analysis of a class (ii) problem without ig- 
noring study of all other possible cases. 

i 

However, if one considers a plant to be parameterized 
by vectors az Of , then the optimization problem, over a, 
would have to be accomplished the cardinal number of a 
times . This means that for every parameter vector in a 
the optimal control would have to be computed and this 
computation performed for all such vectors in (X. For ex- 
ample, if such that (J. = fo,lX, then the optimiza- 

tion would have to be performed ^c. times , where ,c is the 
■power of the continuium. Therefore, one is motivated to 
seek a technique which would allow for the analysis of. the 
optimal problem but which would significantly reduce the 
number of ' computations required if a is allowed to range 
over some set. The most obvious approach would be to hope 
for the existence of an extension of a known solution into 
a neighborhood of that solution. Or in other words, the 
hope is that the behavior of an optimal control u~ corre- 
sponding to a parameter vector a is related to the optimal 
control u— for a in a straightforward manner. For example, 
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L X = [ 0,11 and for any ae (X the '‘optimal" solution of 
w = aw, w(t Q ) = w , is defined to be <J>(t,a). Suppose also 
that it only be required that if a belongs to a small 6 
neighborhood of a the "optimal solution <j>(t,a) belongs to 
some e neighborhood of d(t,a), £ > 0 , <5 > 0 . This state- 
ment requires, for e small, small parameter variations 
result only in small trajectory variations. Obviously, 

4>(t,a) is unique for all ae a . There exists a continuous 

i 

mapping G, such that G: CX E"^x[t o ,T3. But a is compact 
and therefore has a finite open 6 cover. Then, under G, 
the solution space ^(t,a) = |<Ht,a)j- = |<j> (t ,a^ )_| <j> (t,a^ ) is 
the solution of- w = a^w, w(t c ) = w Q for all e |has a 
finite open £ cover. And furthermore, there exists a 
6 > 0 such that for j — Sj <6 and s > 0 given, 

|4>(t,aj) - <{>(t,ou| = 1 exp a^t) - exp {- at) | < e. 
Therefore, instead of considering jC exact computations, 
one need only consider a finite number of calculations if 
the e error criterion is acceptable to the designer. 

The extension mechanism to be investigated will be 
an expansion of the Hamiltonian by a truncated Taylor Series 
about some known solution. In particular, the known 
solution considered will be the nominal solution. The 
nominal solution is that optimal solution which corre- 
sponds to a = a n , where a n is the nominal design parameter 
vector- The Hamiltonian is dependent upon the given plant 
(2-1) in which the plant parameters are imbedded into its 
definition. Therefore, it is desirable, before attempting 
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to extend the Hamiltonian in a truncated Taylor Series, 
to introduce a more flexible notation which will give the 
p.-i r «- r •/••'■tor .i fior<- <■/[« licit. r'-presen tation in the 
Harm J Ionian . Tills is accomplished as follows: 

Define a new state vector (augmented state vector} 
y such that 

y eE n X 71 C E n+M 

where . • 

.y 4 • * , xeE n , azCX. C E™ 

It should be noted that the a defined above is not time 
varying. It was stated earlier that only autonomous sys- 
tems need be considered by virtue of a reduction technique 
displayed in footnote^- This technique is to be thought 
of as a device which will eliminate the -explicit time 
dependence from the plant. For example: 

Consider the -nonautonomous system 

X = (ct^t+o^sin t)X + u te [O/lJ 



then under the transformation t = * n+1 = *- 2 



The parameter vector of this autonomous system is 
( “il 

<t ' where and a 2 are constants. As a practical 

consideration, one may wish to allow a weak parameter time 
dependence to exist. This may take the form of a drift or 
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parameter aging. This situation will be considered in 
more detail in Chapter 3. An example of this proposition 
may be an extension of the previous illustration for 
a^(t) = and (t) — , then 


l 

\ 


ca ( '(t)x 2 + a 2 (t) si nx 2 )X, + u. 

X 2 (t) 


1 

— _ 


Now, also define ! 

*ol 1 

where a Q is the initial parameter vector, which without 

any a priori knowledge of its value will be assumed to be 

the nominal parameter vector ct n . With this notation the 

Hamiltonian may now be expanded in a truncated Taylor 

Series. The development proceeds as follows: 

2 . 3 Extended Hamiltonian Systems 

Define the nominal Hamiltonian to be: 

. .n 

H ty(t) 9 pc-t>,uet.)J = H( p n ctl,u n ct)) 


yc-fc 0 ) s 


for 


pC- 1 ) € 




y , p n (t), u (O being solutions ot (2-^? 
5^ and (2>) -for G, = CL^ s. n d 


y n (t ) = 



- 

-fcx n c±), a n ct>; a 




0 


- 


- 


yet) . 

rx n i 

A <3 


r^o] 


[~6rl 


[a n J 
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‘Theorem 2-1 : Let H(y{t), p(t), u(t)) and all its partial 


derivatives up through order k be continuous in some 
neighborhood N of (y n (t) , p n (t) , -u n (t)) . Then for 

(y (t) , p(t), u(t))eN (dropping the t dependence) 

k-j 

p-p n ,a-u."),V> H(y )Pj u) + 

L « o 

+ u-u n >, V> )A\ y,p,u> 

k 1 

where- . . < (2-7) 

H ^,p,a) s 

e N 


and V is the gradient operator"^. 

The vector (y^ ,p^ ,u^) is a point on the iine segment 
connecting (y;p,u)eN to (y n ,p n ,u n ) N. Because the neigh- 
borhood N is a convex subset of E 2 ( n+m ) +r (i. e . it is a . 

2 (n+m) +r dimensional ball). The interior of N,i(N) is 
either convex or empty. If i (N) is not empty, then the 
following is true: 

Given two points . (y ,p ,u) and y n ,p n ,u n ) in N 
with (y n ,p n ,u n ) obviously in the interior of N 
then every point on the line' segment between 
(y ,pu) and- (y n ,p n ,u n ) (with the possible exception 
of (y ,p,.u) itself) is an interior point of N. 

Because N is convex, N = co(N) (co(N) denotes the convex 
hull of N).' So equivalently, (y^ ,p^,u^)' e (co (N) ) except 
for the previous noted exception. 
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By (2.-7) , the Hamiltonian can be represented in a neighbor- 
hood N of a particular vector (y n ,p n /U n ) by a truncated 
finite series if the Hamiltonian satisfies the conditions 
stated in - the theorem. The last term on the right hand side ■ 
of (2-7) has special significance.. This term, which is 
evaluated at some point in N, (not necessarily (y n ,p n ,u n )), 

represents the error in approximating H(y,p,u) by 
k-i 

;S p-p\u.-u" ), v> w 

i-o 

The term ,, 

-L «y-y" ) p-p",u-u"),V> 
k ! 

shall henceforth be referred to as the truncation error 

v 

and denoted as o(s )‘. The argument e will later be used 
to denote the vector difference between (y,p,u) and 
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(y n ,p n ,u n ) . The necessary condition that H (y ,p ,u) sC k (t Q ,T) 
is really not to sever. In the cases to be investigated 
it will aiways be assumed that H(y,p,u) does indeed 
possess k continuous partial derivatives; If it does not, 
then there still remains another alternative. From the 
Theory of Mollifiers a function can be approximated, 
in norm to within an arbitrary error s, s > 0, by a' 

C jt^/Tj function. Therefore, even though the Hamiltonian 
may not have a desired number of partial derivatives 
existing, it can be approximated by a function which has 
the desired derivatives. Furthermore, the derivatives 
which do exist in the original function will also be found 


in the approximating function. 

The study shall now concentrate on a special trun- 
cated Taylor Series representation of the Hamiltonian 

^ t n 

HctjjPoU) <(y-y M ,p-p^’a-u. n ),V '> 1 


+ 1 p-p n s ^ V> H a.) 

■3 ! 


( 2 - 8 ) 


Therefore, (2-7) is being considered for k = 3 and 
H(y,p,u) e C 3 [ t o' rT j at least, in some neighborhood of 
(y n ,p n ,u n ) . This property will be used later for purposes 
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v^y 1 " 1 *1 

O H<y,p,tn _ U . , K 
- 7 •■•••'“- " 1 'q'p a 

-W W >. y 

where j + i ■)- h - k . 

Then the necessary conditions for the minimization of 
H(y,p,u) given by (2-4,5 and 6) become 

: 6 H ( tj,p 3 u.) 

i p 

= Hf,«j,p,u) + H pS (y,p,uup-p") + (29) 

■+ H py^y^Po u.)( ^p 0 (_(y^p^u.Kix-ix n j)t 

+ Op C£ 

= -H y (y,p,cU - Hyety^u.) <y-y n > ^ (2 _ 10) 

~ ^yp^yt)P,«-)Cp“P n ^ “ /-l ya C Lj, p,a'^CU.-tL h )4- 
- OyCe 3 ^ 

and where u* in (2-6) is assumed to exist and be unique. 
Also, equation (2-6) shall be weakened for computational 
purposes as follows: 

For H(y,p,u) e C^jt^Tj given, 
the optimal control u* satisfies t 

(+) H u ^p>“-> 
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For some u* tNCi(il)*. 

It shall also be required that the second derivative 
type test H u 2(y,p,u *} > 0 thereby insuring a minimum. 

It should be noted that H u (y,p,u) and H u 2(y,p,u) exist 
over N. • 

Summary of Assumptions : u*ei(Q) 

i) u* is obtainable from H u (y,p,u) = 0 
ii) H u 2(y,p,u*) > 0 

By the assumed uniqueness of u* , u* is a globally 
optimal control over N. 

Therefore, u* satisfies 




u. = u 


-t H u e ftjj p,u)(a-a)-f 


* p,t4Hy-y n )+ H^(y,p,nO Cp-p” ) + 


to a C£^ 


* s ° 

over t<£ 

£ T 

i. 

u = u x 

L 

* 



The assumptions placed on u*, that is (i) and (ii) , are 
really not too restrictive if the class of control func- 
tions to be implemented possess some natural smoothness. 
However, such control strategies as bang-bang control 
would naturally be excluded from this analysis. 

From the extended canonical equations (2-9, 10, and 
11) several important properties pertaining to 'the 


If u* satisfies (2-6) and u*e-3fi, then finding u* by 
setting t equal to zero would not make sense. Therefore, 
it shall be required that the u satisfying t belongs to 
i ( ft ) . 
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extended Hamiltonian system become apparent. First, it is 
desirable to insure that the solutions to the extended 
canonical equations are well behaved for an arbitrary 
parameter vector a "close” to a n . 

Lemma 2-1 

For a ->■ a 11 , the solution of (2-9) 
n n 

y = Wp ( SbP-> a ) + H p 2 t4,p, LO C p- p n ) + 

+ Cy,p,OL>(y-L j tl ) Wp U (t f,p,U)(U-CC rt )+ 

+ O p C€ 3 ^ 
ce ; Hp p 5 LO 

converges uniquely to y n (t), the solution of (2-10) 

p = - Hy? cLj 7 p 9 uu + 

o * m 

-^ M p ttSiP^ K P*P n) - H yuL Cy,p 5 a) (u-(x*V 

-Oy(€ 3 ) 
ie: " 

converges uniquely to p n (t), 
and the u which satisfies (2-11) 
te: 3 O 

converges uniquely to u n (t). 

Proof: 

By hypothesis there exists a unique control effort, 
call it u (t) , which satisfies the original optimization 
problem with a being an arbitrary plant parameter and 
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at CSC . Remembering (2-9) is an exact equality and for 


(y,u) cN,- tc Jt Q ,t] 


xcb= f (xct),u a (t) i ay 


x”tio)-xZ 


Because for a = u n , f(x(t), a 11 , u n (t)) is Lipschitizian 
in x(t) , and u n (t) (unique by hypothesis), the solution 
of (i) , for x(t Q ) = x q , is unique and was defined to be 
x n (t). The function f(x(t), a, u a (t)) is also continuous 

on JI a . 

|(t,x(-t>,a|t e[t 0 ,T], (X,a)€ n J ( 

Also, suppose |f(x(t), a, u cj (t)[ <M on Jt a •• 

Claim : 

There exists a 6 > 0 such that for any fixed a 

(which implies a fixed u (t) ) with | | (a - a n , u — u 11 ) j | <6, 

a 01 

every solution <j> a of (i) exists uniquely on |~t Q , tJ and 
as a *• a 11 ('correspondingly ■+■ u n from the continuity of 
H u (y /P./u) ), <{> a (t) vx n (t). 

Proof : 

The proof will be developed locally then extended. 
Choose X sufficiently small so that 

r={u,* )e r"^! it -t-„i <x, ix-x„i * m x , 

where M is a Lipschitiz constant! C N 


Then, for any a, u^cN a solution of (i) exists on 
[t - t ] < X. Let ^ be any solution of (i) for a 
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* 

p-i came Lor vector a and (corresponding contro.1 u . Then 

U 

{(J) } forms a bounded equi continuous family. That is, 
for each e ->■ 0 there exists a 6 > 0 such that 

I l<P a . (t) - $ a (0 || < £ 

16 

where jt - t"| < <5 . This implies, by Ascoli’s Theorem , 
(4>a^} has a uniformly convergent subsequence { ( k ) ( • ) } * 
But from the uniqueness of the solution at a n ,u^ = u n , 

* a ( k ) " v = aS “ (k) * a ? 

This gives' a local result over 1 1 - t | < A. Extend the 

results to ^t ,T^. Notice also-, oniy the uniqueness of 

<J> at a point was required. 

# (Claim) 

Thus, from the claim the solution of (i) tends to the 
unique solution x n (t) as a ot n . Obviously the solution 
of 

a (t) = O', a (t ) = a 
tends to a n as , a -»• a n . 

Now all that remains is to show p (t) -*■ p n (t) as a -* a n 
(ii) p(t) = - H (y(‘t), p (t) , u ( t ) ) , p (T) given and 
H y <y (t) , p(t), u(t) is Lipschitzian in p. Therefore, there 
exists a unique solution of (ii) for y(t), u(t) given. 

Let y (t) , u(t) assume the role of the parameter vector a 
in the claim. Then, from the claim 

p(t) -> p n (t) as a -> a 11 


* 

Fundamental 


Lemma 


13 


<j) is a 


q (t) 
<P (t) 



t 


o 


Caratheadory solution of 
on Te(t o ,t) if and only if 
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The previous results can be used to show how the 
cost index C (u) reacts as a -> a 11 . The obvious question 
to ask is "does C(u) converge to C(u n ) as a ->- a 11 ?” 

Theorem 2-2 : As a -> a 11 , for 'a, a n £<3t,C(u) ->- C(u n ), 

where u denotes the unique solution of (2-11) .for a = <* . 
Proof : 

All that need be proven is that x{t) > x n (t) and 
u(t) •* u n (t) , where x is the solution of (2-9) for a = a. 
Lemma 2-1 satisfies this demand. Then (x,u) ■+ (x ,u ) 
as a -> a n , thus C(u) -*■ C(u n ) 

# 

What of the case where a ^ a n ? If the optimization problem 
is to make sense, one would intuitively expect the follow- 
ing to be trivially true. 

Theorem 2-r3 , mine (u) = C(u*) < C(u n ), where u* denotes 
the unique-solution of (2-11) for a* f a ' . 

Proof : 

Suppose not. Then there exists a u n £N such that 
C (u n ) < C {u* ) . 

But, for (y*, p*, u*) cN the min H(y*,p*',u) = H (y* ,p* ,u*), 
uel(f2> 

# 

It should be noted that a weak inequality is used in 
Theorem 2-3.- Even if u* is a globally optimal control 
over N, it is possible to construct a cost functional 
whose arguments x(t) and u(t) are independent of parameter 
variations, when x{t)£E n {t) and u(t)£ E r (t). 
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For the sake of notational convenience the following 
notational convention shall be established. 

Let 

. n n „ 

Ay = y - y ? y,y eN - 

* n n r , 

Ap = p - p ; p ,p eN 

• Au = u - u n r u,u n eN. 

Also, in (2-9), (2-10), or .(2-11) it should be noted that 
H ^(y*P/U) = 0 and Hp 2 (y/P/U) =0 (due to p appearing 
linearly in H(y,p,u)). From (2-11), Au may be explicitly 
solved in terms of Ay and Ap because H^ 2 (y>P/ u ) > 0 by 
hypothesis; therefore, H^2 (y,p,u) _1 exists. 


" | [^ py ( y»p.u) Ay + 


^ U.p Ap 


0 u.t€ 3 ;>] } 


(2-12) 


The results of (2-12) may now be used to eliminate Au from 
(2-9) and (2-10) by direct substitution. 


^ pLj £ *-j ■> pj CL) f~^ pu. ^ ^ ) p o CL 5 i p) ) 

Li" , 1 fLj M (2-13) 

Cy,|D,u.J J4y - [ H pu. 

[f^u z p» ^u. P 

" (L h p> \_^u Z ( V’ P> “->} °U. C€ ^ + °p C & 
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where: Ay(t ) = -- , 

o LyJ 

■y arbitrary. 

The arbitrariness of y will not affect the desired result 
’as will be shown shortly, and 

4.p = ^ y 2- ^ y U~) ~~ Lju. ^ ^5 jOj lx.) H uZ C p, to ) 

W U y (y,p } 00 “ Wy a ty,p,Uj 

~ (I n . ]-« u ° ~j (2-14) 

H ( y j p,u.) ‘’exp o p? ^ P 


■ W yu /y,p,«-> J-l u z (Lj,p,a.y o u ceV o y W 3 ), 

^.pCT) giuen; 

'For N sufficiently small such that the higher order terms 

( e 3 ) , Op(e 3 ) and o^Xc 3 ') are negligible, then (2-13)' and 

(2-14) can be represented as a system of 2 (n+m) linear 

ordinary differential equations. (Error bounds on o^e 3 ), 

o (e 3 ) and o (e 3 ) will be established later). Suppose 
hr y 

such a non-empty neighborhood exists. 

Then : 



i_ &ct J 

cay i -A'a) 



Get) 


a p 


(2-15) 


Ay Q , Ap(T) given 

where A, B, and C are defined in the following manner: 

^ 

For H(y,p,u) = L(x,u) + < p, f(x,u,a) > + < p, 0 > 
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Then, from (2-13) 
- p n 


/ 


A ct) * ( 


p n i p" 

+ x ( x, u,a ) 1 w x, u,a) 


o 


■ o 


^cx,u,a)TL^2tx,a) + 



p y 

n~ 

-1 

", n 

rp > - 

+ 

T lx CX,U.,CL) 

u. 


L {x,u)-h 

ux * 

t UL (x,u,ajp_ 


n 



^utx,u,ai pj a j j 
'f . (X. u a)l r 1 U 7(X,U) h [(^a,uA>p]J 


U. 

" : 

j 

X 1 

j 

(2-13i) 



f u cx,u,a>;o 

i 

(2-13ii ) 


1 


X <x, u,a) p 

ri 


1U- 


!a 


r ^ 


•f 


X <x, u,a)p 
o 


n -I 

a; 


+ 


+• 

(2-14i) 


•| L (X / cc,a)p 

*-0 ex, u,aj eCIC^ ,1) 

[fa^ a " a) lu 

^J(x,u,a)p]3''[L;o ; ,a^[ S[.^«,tt.a)p]"]} 

A rather interesting phenomenon occurs in equation. (2-15). 
The -rows of .its Act partition are equal to zero. In fact, 
the truncation errors associated with the A& rows are also 
zero. This implies that the autonomy of the parameters 
have been analytically preserved. Therefore, the parameter 
variations considered, of the form a — a , equal a con- 
stant. For a physical viewpoint, however, it is desirable 

to allow for a previously noted weak time varying of the 
parameter. If so, such qualities as parameter drift and 
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aging may be considered. This quality shall be introduced 
into the problems structure as follows. Assume the exist- 
ence of a 5 (t) which will appear in the Act equations in 
place of a truncation error. Then A& = ' K (t)- ,Aa (o) 
arbitrary has a solution which is equal to a constant plus 
a low amplitude, or slowly varying time dependent term if 
| |s(t) j [ is small. It will be demanded that | |f,(t) | j be 
sufficiently small so that it may be dismissed as were 
the legitimate truncation errors. This is a reasonable 
demand because one would not expect' a set of parameters to 
exist which are rapidly changing or strongly varying in 

magnitude^ to admit a truncated Taylor series represents- 

• \ - - . , 

tion of H (y ,.p ,u} . 


Eq . -(2-15) is recognized to be a -matrix Riccati type 

* t 

equation. it can be recognized' as such because of the 
structure _of G(t). The solution of (2-15) is given by a 
(n+m)x(n+m) nonlinear system of differential equations. 
That is, there 'exists a symmetric K(t) such that 


*•' - icKi = A 7 ob) K t-t) + .K(i)'Att) - K(t) B(£) Ktt) + C tt) 


KCT) = 


T O 

o o 


(2-16) 


and Ap (t) = K(t)Ay(t). 


The solution of * will usually involve a computer 

mechanization of the problem. Some techniques of solving 

17 18 

a Riccati equation. can be found in Tuel , Rriedland , 
and Bass with an example supplied by Stoner, Taylor, and 

1 9 . 

Bass . If any of the 2 (n+m) solutions of- (2-15) are 
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known in closed form, then partioning routines can be 
used' to reduce the computational difficulties associated 
with finding K(t) 13 . Partioning techniques also 'suggest 
another method of realizing the elements of K(t) for this' 
particular .problem. 

Partition K(t) follows: 


Ki-fci 


K , Ct) 

! K 2 Ci)" 

' K' 2 Ct) 

1 K 5 ct) 


where is nxn, K 2 is nxm, is mxm. 
(2-12) can be expressed as 


' Then Au defined by 


Au 


n 


Mu 2C H*P> u) 


r.-.n 


H 




utj pj W-) + 

Ay 


- n / 

(2-12i) 


Therefore, one needs only to focus attention on 'calculating 
K^(t) and (t) . From (2-1 3i_, ii) , (2-14i) ', and (2-15) 
the differential equations defining . K^, (t) arid (t) are 
established, as: 


Act) 




A t is n*n , A z is nxm 
A 3 is mA-.n, A 4 is mxm * 
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A 3 /■£.) 

6 ti) = 

B 2 a>- 1 
Cct^ = . 

Cj(t) - 

r 

..] 

C 2 (-fc) = 

. [ 
C 3 ct)^. 

+ 

C 4 (t) = 


" f-t) = o 


B, (rl> B 2 U) 
B 3 . (.-L) . B 4 (t> 


, Bj (t) is hxn , B 2 (t) IS nxm 
B 3 a) is hoxn , )5 mxm 


4- u c x,u,a)[ l^ 2 ex, u) +( f a (x,u., co p£.r , ‘4«>“»a) 


5 3 (i)= B 4 /£) -o 


D I 


Cflt) 15 nxn ^ ft) is n xm 


C 2 (t); 

LC 3 ci.) Cyt) ,s CfCt) /s mxw 

V a ’ ul+ 'f(“' a ’ a, .P)] *• [L n Au «i“) + 
h (4«,u,a}p)J[ L ” u 2 cx,tu +(IJ cx,u,a)pfj' 
L n ^(x,u.) + (-f u (x,u,a) p)x ] 

;L u hx,u) + (l u (x,a,a)0'(4 / a,u ) a)pf 

-/ 

.(r a a,a,a)p)^ ~(-f- a (x,u,a ipfQjtXiU) + 

iii 

(4x^,u,a)p)J (L ux (x,u) + (~f u oc,u,a>p)*) 

(■f a (x,a, a ) p) R [L^ cx,us +-(-^a,w,ajp)”]" 1 
{£cx.u a a>p£ 
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Therefore : • 

«> - K, tt). K,rt)A,rt) K.t-ti - K, rt) 

+ C, ct) , K,ct)= t 

and 

« 

cu) - K 2 (i)- K, i-t) A 2 <*>-*■ A' i-fcj K 2 ct) - 

+ C 2 (-fc), K 2 (T)= o . 

Comments : 

Equation (i) is a homogeneous nxn nonlinear differen- 
tial system of equations. Its boundary data is specified 
at t = T. It is independent of the initial. data Ay(.t Q ); 

Equation (ii) is a nonhomogeneous (nXin) linear 
differential .system of equations. Its boundary data is 
specified at .t « T. It is independent of the initial data 
Ay(t Q ) , thus leaving Aci q arbitrary as acceptable. 

Consider now the solution to (2-12) to be of - the 

form 

Auii)- ( 2 - 17 ) 

where G(t) is calculated through one of the suggested 
methods. A remarkable observation may be abstracted from 
(2-17) . The implementation of the adaptive control need 
only require that x n (t) need be stored in some memory 
device. The simplest example of such a device would be a 
tape. Thus, after G(t) has been precomputed, G(t) , being 
a function of the nominal variables' y n (t) , p n (t) , and u n (t), 
it will be multiplied in real time by (y(t) - y n (t)) . 

But y n (t) equals (x n (t) , a n ) , and a 11 is nothing more than 
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some known biasing term. Thus only n 'known time varying 
functions must be -introduced into the system from memory 
in real time instead of 2(n+m)+r functions which might 
have been earlier predicted. 

■ The schematized realization of -the adaptive system is 
given in Figure. 2-2, where a is an estimation of a* and 
a* is the actual plant parameter vector. Parameter 
estimation will be treated in Chapter 3. 



Figure 2-2 
Adaptive, Structure 
2.4 Truncation Errors 

To complete this section, the structure of the 
truncation error terms will be investigated. Rather than 
an exhibit the results in tensor notation, the equivalent 
scalar triple sum will be analyzed. 

Let 2 i (y,p,u) E 2(n+,n)+r 
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Then the truncation error of (2-8) becomes 


CO 


gfh+rrO-j-r 

-"V 


.*> b H^cz) 


tj*»i • „ L J 


Evaluating b^ H ^ Cz ) for special cases 
. bz .1 &zj &ZK- 

it can be noted that a and p appear linearly in H(y,p,u). 


Therefore,, for y = (z. ,z\ ,z, ) having any two or three of 

1 j K 

its element being an element, or elements of p, or the 


last m elements of p, 


bz z b'Zj bz K 


o , 


For A == z.,z t ) having two or three of its elements 

■ ^ 1 j K, 

being an. element or elements, of a 

b H ** c z > _ c «, 

It may be noted that the truncation error, defined in (i) , 
is dependent upon z^,, z g eN. The requirement of finding 
such a a* which establishes the equality in (2-8) may be 
relaxed by redefining the truncation error. Let the 
bound on the truncation error, denoted as e , be defined 
as follows : 


(ii) 


£ a 


X 

3 } 


r 2 (M -n» + r 

E 


sup 

Z€N = I 


CZ I -zfXZj+zfXZK-ZZ) ^. HtZ) 

dz.dz jdz, 


> sup 

2£ N 


E tz i -zrx2j-z]'xz K -2K)^hV)_ 

OZibz] oz 
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Also, the truncation error associated with (2-12), (2-13), 

and (2-14) may be interpreted similarly. 

Let' Tj (z) be the middle term of (ii) . 


Then 


r 


( iii ) 




O a (€ 3 ) < sup 

Xg{§? } 

o c € 3 ) < sup r \ 

y ^ N l5Ti 






y 


Therefore, requiring o (e 3 ), o (e 3 ), and o (e 3 ) being 

- ^ p y 

small can be insured by requiring, the right hand side of 
(-iii) is small. ' 

2 . 5 Example Problem I 

• ! 

Example: --Linear Regular’ (single channel control) 
Consider the plant 
(i) X(t) = A(a)X(t)+Bu(t) , X(t Q )=C 
and the cost index 


(ii) C (u) ■ = %<X (T) , TX(T)>+3 s 
+ <u (t) ,.Ru (t) > dt 


L 


<X(t) , QX ( t ) >+ 


where 

The terminal time T is specified 
- T is a 2x2 positive semidefinite matrix 
Q is a' 2x2 diagonal positive semidefinite matrix 
R is a lxl diagonal positive definite matrix 
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The nominally optimal solution is found as follows: 
Line the nominal Hamiltonian to be 

H(X(t) ,a n ,p(t) ,u(t) ) - ? 5 <X(t>-, QX(t) >+ 

+3s<U(t) , Ru (t) > + <p (t) ,A(a n )X(t)+Bu(t) > 

and 

(lii) p(t) - QX(t) - A " ( ot n ) p ( t ) 


9H 


3u (t) 


= 0 -> Ru (t) +B "p (t) = 0 


,“1, 


then 

'u(t) = - R J 'B"p(t) 
Note also 


?. 

9 H 


9u.(t ) 


- R>0 


Then 



x n <t> 

/ A ca n ) 

x n a) ‘ 

(iv) 


\~Q. 1 -A'canj 

p h (t) • 

_ 


and P n (T) = TX n ( T ) . 

Assume the solution of (iv) to 'exist and be of the 
n . . i i n 


n , 


p,iti 


p*<«. 


'(<;;£« ! k; 4 <t> 

■kTcoTk" 


■ 22 


(t), 


x"a) 


X"(t) 


■ Now construct- G of (2-15), remembering 





'XjC-t) - 


Pi (t) , ‘ 

<t) = 

x 2 ct) 

r» pCt) = 

p 2 Ct) 


a, 




a z - 


- P 4 (O -1 


De- 


form 
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o 

} 

o 

o 

i ° 

o 

O 

o 

'CL, 

■ a% 

x, n 

X* 

*2 

I o 

-i/r 

/T ,i 

o 

o 

O 

O 

o 

O 

o 

o 

o 

a • 

o 

O 

o 

O 

1 o 

o 

a 

o. 


o 

Pa" 

o 

I o 

a n 

/ 

o 

o 

o 

n 

"^22 

o 

Pa" 

1 -1 


o 

o 

P* 

O 

o 

o 

1 o 
1° 

-V 

-*2 n 

o 

o 

o 

n 

Pi 

o 

o 

o 

o 


Note : 

p^t) could be redefined in terms of X n (t). 

ie: = <X,xM». 

L = l 

Therefore, 



~ o -a? o o 


- O 1 O d> ~ 

k it) = 

i -a^ o o 
O X? o o 

Kho Kht) 

-a?-a n 2 x? x n 2 

O -o o o 


_o x 2 O o 


0 

0 

0 

0 

! 



r O O O O ~ 


--^11 o 
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where 


K(t) 

K(T) 
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Solve for K(t) and construct G(t) of (2--17) . The bound 


n> a 
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on the truncation error for this example is: 


e'= SUp[( a]L - a*) (X 1 - Xj] (p 2 - P 2 ) + (a 2 " a") (X 2 - (P 2 “Pg)] 

zeN . ‘ 

For example, if zeH, N = (x , a ,u,p)' j }| Z || < r j- • 
then e .= 2r 3 . 

Also . - 


o u (e) = ° 


On<<h= 


°y c€) = 




o 

°p 2 (^ 


(a-alxx.-x^+CcL- 

°P 3 C€ ’ 


o 

°P4C€) 


o 

e>x, cei 


ca,-a"xpi-pJ>’ 


°x e ce) 


ca 2 a"xp 2 -p 2 ) 


°a,ce) 


(x. r x?XPz-Pt n ) 


°a 2 '« > _ 


CX 2 -^K,p 2 -p 2 n ) 



Under the 1 previous example N, 


° U C€ ) = o 


O p (€ ) = 2 


OyC^ 3 ; = 2 


0.1 
r ‘ 
o' 

O J 

- IT 2. — i 


L k- 2 J 


Example Problem 2 : 

Consider the plant 

X(t) = oiX(t). + u(t), X(0) = 1 
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A*'- 1- Um i)omLri.)l value- of .1 ha ,t n « ~ I. Al.so, consi dor 
the cost index given by 

Ccuo^ ~ f ?x 2 tt) + bt 2 <t) dt. 
c Jo 

The nominal system is optimized by solving 

X n (t) = - X n -(t) - p n (t) 
p n (t) = - X n (t) + p n (t) 
u n (t) = - p n {t) = - k n X n (t) 
where k n is the nominal Riccati gain and is equal to 

k" = -bfe > 0 

Thus p n (t) = (- 1 +^/l)X n (t) 
u n (t) = .(1 -y[2)X n (t) 
and X n (t) = exp(--\/it) 

Now consider the system subject to the parameter variation 
a = a 11 - e , u(t) = u n (t). 


oxct> + > x n tt> 

= - d+ €)x(t)+ (i-y/i) exp C ijz -t) 

/V' 

X(O) - ] 


X c-fc) = exp 




“V^ >+] ' 5 } , 


( i ) ( 

■ f i + e ~Jlf 


ex p 


[C i + € 


Note: for e' = 0-, X(t) = X n (t) 

Xi 

For a = a - c = a * , the true optimal strategy is 


X*(t) = - <l+e)X*(t) - P*(t), X*(0) = 1 
P*(.t) = ~ X*(t) + (1+e ) p*(t) 
u*-(t) = - p*(t) = - k*X* (t) 
where k* = - (1+e) + y(l+e) 2 +l > O, 
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and 


[ -V 


1 


X*(t) -= exp I - y (1+e) +l)t 
Now the adaptive dynamics are: 

for a , = - 1 , a 2 = X n (t), =f 1, c, = 1, c 2 - p n (t) 
Therefore . 

k e C“i)4-(-i)kf- kfcokfti =-kf 

But this is a homogeneous constant coefficient differen- 
tial equation evaluated over t = 0, T ->■ «. Therefore, 

& 0 

from a steady state argument k^ = 0. Thuh 

kf = = k". 

Also: 

-k* = kfx"- Y-t KK. * p" » K e 2 c-t-«u = o. 

Integrating backwards in time one finds 

k. e it) = 2 c 

Therefore 
c 

U 


fore 

L e tt) = uh-t) - [( - I + ^2 XX- X n ) * ex p C-fi t)2 ( |.yi Ma] 


Tii" i f n n n 

Letting Aa =' a - a = a - e ~ a = -e 


X € ct)~ -n 


+• OX^Ct) - )X e (t; - exp -fe) 2 C-f+^2 )€ 


Thus 

t 


x e (-t)= CXp (-^2 -e)t - exp[- 0^2 + 


2C-J+^2)€ 

ex p[- c {z- + e it] { I - - 1 - { E 5 [ e ^ w " '] } 



Notice/ for e = 0/ X'^{t) = X“(t) and for et small 
X e (t) "S' x* (t ) . Consider now a 10% parameter variation 
(i.e. e =.l) . One finds 


-9 V 


■8 A 


xtfc) 77 


X*ft> 

x<o 


X (\) = C (. 796 ,) 

x*cn *e /m/ c.&80') 
x e cn= e L> c.T 2 o) 


Figure 2-3 
STATE SPACE e = . 1 


Now considering the various control efforts considered for ■ 
e. = 1 .. 


u*(f> = < Li -fiJl } exp(~-$>~2l t ) 

U n (i)= (- 414 ) cxp(-fe t ) 

art)- u n «H(-t^5 >a e rt)-x n rtfl-o<P^ t ) i f- |+ # )( '^ 
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1 2 



From <+> it can be noted for (st) small (i.e., the. internal 
where X e (t) has its largest values) X e (t) acts optimally- 
The non-optimal characteristics of X (t) occur when (et) 
is large. But over this internal X e (t) is small and its 
contribution to C(u e ) is minor. 



Figure 2-5 
GAIN PLOTS 


It is easy to see from Figures 2-2 and 2-3 that 
C (u* ) < C(u e ) < C(u r ) 

In fact. Figure (2-6) exhibits a significant reduction in 
cost by using u e (t) over u n (t). 
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Figure 2-6 
COST INCURRED 

2 . 6 Reoptimization 
It -is assumed: 

(i) that the actual state X(t) can be measured in 
real time in the absence of noise. If the observa- 
tions are noisy, then a state estimation technique 

•might Have to be considered. (Examples — nonlinear 
stochastic filters, Kalman filter, etc.) 

(ii) x n (t) and a 11 can be loaded into the system in 
real time. This could be accomplished by loading 
these vectors from a tape which is synchronized in 
real time with the physical system. 

and 

* @ 

(iii) a can be generated (estimated) . 

Methods which satisfy (iii) (i.e., the parameter vector a 
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need be estimated) and will be treated later in this 
investigation . 

It may be noted that the feedback loop is adaptive 
and produces a control effort which is in such a direction 
as to minimize the cost incurred. This preservation of 
the concept of optimality will be called "reoptimization." 

In order that the adaptive structure be of practical 
use it must possess the quality that: 

.. jC(u*) - C (u e ) | < |'C (u* ) - C (u 11 ) | 

0 

for u being the adaptive* control and for a set of 
parameter vectors in some neighborhood of a 11 . 

Theorem 2-4 : For an adaptive control u e satisfying (2-17) 

u 0 £i[n) 9 there exists e > 0, and sufficiently small, such 
that .for ;( ja e - a*| | < e; a e ", ae CX r 

| C (u* ) - C{u e )j < j C (u* ) - C(u n )|. 

Proof : 

For a = a* ^ -a 11 , C(u*) r C (u 11 ) from Theorem 2-3. 

From (2-13) and the ma.tricies defined by (2-15) (a e being 
the approximation of a*) 

(*) y* - y® = A (y * - y e ) + B (p* - p e ) + 

+ y.(y* - y 11 , p* - p n , u* - U n ) , 



where y (y* - y n p* - p n , u* - u n ). is the truncation error 
of (2-13) for (*) . From (2-14) and the matrices defined 
by (2-15) 



47 


P “ P = C(y - y ) - A'(p - p ) + 

+ y(y* - y n r p* - p n , u* -'u 11 ), 

(p (T) - p e (T)) = given 

Now choose N ^ <j> such that the truncation errors y and y 

xi n jrx 

are neglectable. Either N is the point set (y , P , u ) 
or otherwise. If is the point set (which is the result 
of insufficient smoothness of H(y,p,u) about H n (y,p,u)), 
.finished. If N £ is hot the point set, then there exists 
a e such that J | a* - a e | | < | Ja* - a n | jv 

• In an argument similar to that of Lemma 2-1, except 

e * . r* 

ct ->■ a instead of a ; 

■ - i .e . : 
the solution of 



tends to the trivial .solution (i.e., (y (t) - y e (t)) -+■ 0 
* e • 

and (p (t) - p (t) ) -*■ 0. The trivial solution is the unique 

t . 

0 * 

solution of (i) for a = a (i.e., £ = 0) . It is apparent 
0 * 

that u tends to u for this local argument because of 

their linear structure. 

* 0 .0 

Therefore, for a and a eN £ and a sufficiently close 

'ft 0 0 * 

to a , L(x ,u ) defined in (2-2) can be made arbitrarily 

close or equal to L (x ,u ). But u is a fixed function of 
• * n 

time and for a ^ a is some constant non-zero vector X. 

Consider the sequence of parameter vectors a?, 
a^eN , such that | | a? - a n | j =0 for i, = 1, 2, ... . Then 
for a and a^£N £ , and | |a - a^| j = X, L(x^j u.^) need 


P e CT ) « p ! 


VI ) 
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not converge to L(x ,u ) as i + ®. Then | |c(u ) - C(u e ) j | 

< | fc(u*) - C (u* 1 ) [ | for a set of a e s sufficiently to 
* 

a . 


The adaptive structure now formally possessed the 
qualities demanded of a reoptimization scheme (see 
Theorem (2-2), (2-3, (2-4)). Namely, C (u e ) + C(u n ) as 

a e ■* a n , C(u ) < C(u n ), and C (u 11 ) -> C(u e ) as a e -* a . 

Also, subject to the constraints imposed on the problem, 
the derived adaptive control is the "best" minimizing 
control for the given problem. One intrinsic feature still 
needs attention, namely the computational requirements 
imposed by the necessity of generating a (see Figure 2-2) . 



CHAPTER III 


PARAMETER ESTIMATION 
3 . 1 Parameter Estimation. 

The objective of a parameter estimator is to approxi- 
mate the actual parameters while hopefully satisfying as 
many of the following qualities as possible. 

Qualities to be achieved: 

(1) Compute uniquely the unknown parameters. 

(2) The estimation should converge rapidly to 
an accurate solution. 

(3) The computational routine and its physical 
mechanization should not be complex. 

(4) The parameter estimator should be free of 
dimensional restrictions. 

The efficiency of a parameter' estimator may be thought of 
as a trade-off between the first three qualities. Quality 
4, however, is an often neglected property of estimation 
techniques and will be given special attention. After 
all, any method used must allow for all the desired 
parameters independent of the'ir number of be computed. 

* 

Therefore, the estimation device is truly an engineering 
problem. 

If the plant parameters can be obtained by direct 
measurement, then the reoptimization problem is direct. 

In general, however, one cannot hope that all, if any, 
of the parameters are monitorable.* The estimation of 

* 

Monitorable implies obtainable by direct measurement 
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non-monitorable parameters will vary in complexity and 
accuracy as a function of the computational technique 
considered- It is for this reason that numerous selected 
computational techniques will be developed leaving the 
choice of which method to mechanize a matter of personal 
preference. 

To motivate some of the numerical and gradient tech- 
niques to be developed in this Chapter, the effect of 
parameter variations upon the systems' trajectories will 
be explored. With suitable modifications, some due to 
H. Hermes, a method 'Similar to one- developed by L. S. 

Pontryagin, V. G. Boltyanski , R. V.. Gamkrilidze, and 

20 • . 

E. F. Mischenko will be employed to examine the 

parameter-trajectory variational properties. ( P on-try agin 1 s 
work was used to show how a given trajectory was. dis- 
turbed by variations in the control effort.) 

Suppose x(t) and u(t) are monitorable, u(t) measurr 

able (in the sense of Lebesque) , then, a can be estimated 
as follows: 

Consider 

x(t) = f(x,u; a .) (3-1) 

’ X <V = x o 

a not monitorable over te[t ,T] 

Assume the solution of (3-1) exists over tef^ - is, 

[t Q ,T], 


£ > 0 
e > 0 . 
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Consider the model of (3-1) to be: 


X e ~*f C* e ? u c ,G c ) 
X e (t,-JU) = X 


(3-2) 


t e Ct r ^ e ,t,3. 



Figure 3-1 

Trajectory Variations 




For x e (t) , u e (t) measurable, the game is to find an a e 

such that (3~2) models C3-1) in that it minimizes some 

* . e 

normed difference in their trajectories . Define 
to be the solution of C3-2) for a = a®, a?e a , or i 
being the cardinality of a . Define: 




r e (t>- U*f 

rat s a 

C 

Obviously 

x it,) Dll co* 


Figure 


3-2 


re(t 1 ) 

; 2 

An E norm will be used. While a L [t^ - norm 

would be desirable, the computational problems become 
prohibitive for this development. 
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Let it be demanded that is a Lebesgue point of a, 
measurable (i.e.: if 

■fc ( 

/ t c y, a" >dt= + OC£> 

t,-ie 

where o(e) denotes "order of e") . 

Define an elementary perturbation of a as follows: 

af on -fc,-i€ - 'fc * t, 
a* eis«<-o/7Cire 


Sr. = 

t 


a 


(3-3) 


i 


A X 

Consider ( t ) => x (t,e), u(t)efl and assumed known (i.e., 

monitorable), and x. (t,e) = x(t,a ). 

• i 

Now compute 4>(o) as follows, using (3-1), (3-2), 

(3-3) , 

Urn 1 [x5c-t., 

£ -*-o € 

t| 

- Jim 1 f cfcx e (t,o,a?)- 
^ im "e { ■Pcx(-t,),a*)Ji€ + oC€) | 

(3-4) 

- = [ -f<x<t t ) 9 af )- fcxct,),a*)] J 


. <£> (05 
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Define (i) 


Kr 


d x (t,,e) 
di 1 


(3-5) 


l€ = 0 


(ii) 




GL,. close to CL 




Then 


x (t.jG) = x ct.) - ct,)e + oc€"> , 

g CXjj.^<= Ot 7 


(3-6) 


(3-7) 


is a point function. 

Let a(t^) be a hypersurface of all solutions of x = 

f(x,u;anx) at t^-^. Notice also (t^) is tangent to CJ ( t 1 ) 

at the point x (t^) . • 

i 

If o(e) is sufficiently small 


X TT: (t ,.£) = (3 - 8) 

V M t. 

The error in approximating x(t,) with x (t, ,c) is 

x n £ x 

(t-^) • Minimizing | | (t^) | | for e fixed and suffic- 

77 i i 

iently small, one achieves by using (3-7) , 

min li 6 )il - min II [-fcx/fc^a*) -"f cx,t,,af ]i. ii 

„ p 0 

which can be written for a minimum occurring at aZ, = a as 

rnin ilix^ (!,,£ )ll = mtn ii [Xit.J-'ffX It,) ; af 1 JL H 4 (3-9) 
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The rCriuli of ( 3—')} Lend:; LLr;ol.f.‘ Lo on intuitive 
interpretation based on tricingl.es. This oversimplifica- 
tion will investigate the solution to a two-dimensional 
system' of .ordinary differential equations. 



Let b - a = 2., c - b = v and the line segment ab represent 

•k 

the solution (3-1) (i.e., a = a ) and the line segment ad 

represent x (t) for a = a . The slope of the line 
U i 11 i 

segment ac is approximately that of ad (i.e., (t) ) . 

Therefore, as v 0 


O 
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Several engineering problems must be resolved if (3-9) 
is to be developed further. They are: 
i) Is x(t^) monitorable? 

ii) What are, if any, the dimensionality restrictions 

.on the problem? 

♦ © 

If x(t) is monitorable, the a which minimizes (3-9) 

may, in some cases, be directly computed. The vector 

elements x^ (t^) , 1 <..i <. n, which are hot monitorable 

must be calculated from monitorable information. Some of 

the methods which will facilitate this are: 

a) One-sided derivatives for x(t) sufficiently 
smooth and He sufficiently small., I pro- 
portional to an a priori smoothness judgment 
on x(t) . 

b) If any element of elements, x^(t), of x(t) 
are independent of parameters, x^(t) can. 

be computed directly. (i.e. , x^ (t)=G(x(t) ) , 
x(t) monitorable) 

c) Mesh refinement methods where observations 

are obtainable at times t^, t^s [t^-et , t-^j , i = 1, 
..., h, t^ monotonically increasing. The follow- 
ing methods will establish derived derivatives 
at a point, or points interior to [t^ - eH, t-^] . 

(1) Derivative formulas from difference 
operations . 

(2) Central difference formulas [21] . 

(3) Modified Euler’s Formula. 
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(4) Weighted averaging of a sequence of 
Xf ( t i * g ) whe re xCt i ) i s found by any 
applicable method. This will smooth 
the data and reduce the effect of 
data points which have a large 
variance from the mean. 

The. techniques of finding xft-^) will not be pursued 
any further. The problem of numerically approximating 
a derivative is that it may -be noisy. Other approxima- 
tion methods shall be developed which will not have need 
of a x(t^.) computation. Therefore, it will be assumed 
that if x(t^) is a required computation, it may be ac- 
complished with sufficient accuracy so as not to intro- 
duce significant errors into the system. The question 
of dimensional requirement will now be treated. 

3-2 . Dimensional Restrictions 

In Equation (3-9) it shall be assumed that xft^) has 
been satisfactorily computed. Also, x(t^) is known from 
direct measurements and the mapping f of (3-1) is well 
defined. Then the problem of finding an a which satisfies 
(3-9) becomes a problem of solving a linear system of 
non-homogeneous equations. Because a appears linearly 
in (3-9), x(t 1 ) , x(t^) being known constants, one is 
interested in the solution to an algebraic equation of 
the form: 

/A(i,)a e = b<t,) 


(3-10) 
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where b(t^) is a n vector whose components are 
i = 1, . .., n, and b^(t^) = x^(t^) plus any term in the 

ith row of f(x(t 1 ); a e ) which does not multiply an 

0 ^ 
element of a . The matrix ACt 1 ) is n x n. If there 

does exist a zero row in A(t ) remove it. by reducing the 

dimension of A(t-^) and b(t^) by some appropriate amount 

(all remaining arguments will be applicable) . The matrix. 

A(t^) is formed by those fixed lumped system parameters 

and known state variables which multiply any element of 

a e . The construction of ACt-^) will have at least one 

entry per row. For example: • 

+ /(j \ / X, t't ,) O O o \ 

a z x\ + *3 Act,) =| 0 o o j 

a 3 x,+ a 4 x 3 / Vo o x.ct,) x 3 (t t )/ 



A matrix A(t^) which has only one entry per row is 
exemplified by 



^ i X i + X 2 \ 

a. 2*1 + k 3 

^■ 3*1 + X 3 / 




Let the rank of A(t^) be q. 


A tt,) - 


x,(t,) o o \ 
o o 1 

o . O / 


A(t^) will be assumed to be free of zero rows. A zero 
row would be the result of some b. (t) = 0 in (3-10) being 
independent of a e . 1 
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If q = m = n the solution of (3-10) is trivial and 
is 


a e = A ^ (t., ) b (t, ) 

If q = m<n and xCt^) is monitorable , and x(t^) is monitored 

and/or computed exactly, (3-10) represents a system of m 

consistent linearly independent equations. Then construct 

A(t,)a e - b(t,) 

where 

A(t,) 15 m * m 4 rank m 
e r m 

a.e L 

y r- m 

bc-fc,) € L 

the solution becomes 

a e - A h,) b it,) . . 

Otherwise, two types of degeneracy can occur. R. E. 

22 

Mortensen listed -them as: 

' 1. If x(t^) and/or x(t^) is not monitored or 
computed exactly, and if b ( t^ sIm(A (t^) ) 
then no exact solution is possible. Im(A(t-^)} 
denotes the image space of A(t 1 ) . The image 
space of A(t^) is the set of vectors 
b(t^)eE in of the form b(t^) = A(t^)a e for 
some a e eE m . Im(A(t-^)) is a subspace of 
E n . Case one is like having more equations 
than unknowns. The special case to be 
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considered is one where b(t^) is considered 
noisy , and one may choose to take redundant 
measurements. "Noisy" shall be inter- 
preted as the uncertainty associated with 
a measurement or a calculation. In this 
case, one may request a solution in a 
"best lease mean squares fit" sense. (See 
end of case 2 for further results.) 

2. If ker (A(t- L )) ^ <f>, then a solution, if it 

exists, is not unique. Here ker (A(t^)) 

denotes the kernel of A(t^) . Suppose E m 

and E n are the spaces to be considered; 

and let A(t^) be the linear mapping 

A(t-^) : E m E n . Then the kernel of 

A(t^) is the subset of vectors a sE 

0 * 

such that A(t^)a =0. It follows that 
ker {A(t-,)) is a subspace of E m . The 
case where ker (A(t^) ) ^ <j> is like having 
more unknowns than equations . This par- 
ticular formulation may easily be 'the 
class of problems that (3-9) is imbedded 
into. However, there are ways to circum- 
vent a type 2 degeneracy to achieve a usable 
solution, if one does exist. One way of 
achieving the solution is with quadratic 

programming with an ordered vector cost 
22 


functional . 
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Because uniqueness cannot be achieved, it is desirable 

6 

to at least guarantee that the computed vector a 
has' properties which make sense in the physical system. 

Let it be required that a e be close to ct n . This will 
exclude solutions which require the system parameters to 
have large variations from the nominal. It is reasonable 
to suggest this because the adaptive problem considered 
local parameter variations about the nominal. 

Let 

A(t,)a e =bt-h,) = X 

and 

II a e - aii 2 =<a e - a", a e ~ a n > 

II XII -<X, X> 

X £ X- A(t,ia n > 
r - min { ii XII i- II a e - a" II } 

This means, first determine an ot e such that | |X| j is mini- 
mized. If b(t^) eIm(A(t-^) ) f then | j X | | = 0, otherwise 
minimize | |lj | >0. If ker (A{t-^)) = 0 then the solution 
is unique arid the problem is finished. If ker (ACt-^)) 0 0, 
then it is possible to minimize not only | | X j | but 
j | ct e — a 11 j | also. This results is a "best" solution with 

respect to the given T. This can be more compactly stated 

22 

by the following adaptation of a theorem found in 
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Theorem 4-1 : The solution to min { | | A | j t-j | a e -a n | j } , where 

A ( t^ ) a e -b ( t^) = a, and A(t^) ,b Ct^) ,a n are given, is a e = 

A ( t^) b ( t-^ ) 4-a 11 , wheife A + is the pseudoinverse of A. ^ 

The notion of a pseudoinverse is not limited to a 
type 2 problem. Consider a type 1 problem again with 
min (i.e.: an overdetermined, or overspecified system 

of linear equations) . 

Define: 

B * II Att,)a e - bet,) II 2 - < A (t, ) a e - b (t, ) , A tt: , )a e - b£t,)X 

The parameter vector a will be said to be the "best 
solution in the least mean square sense" if it minimizes 
2. The minimization is accomplished by setting the 
gradient of j | ACt-^) a e -b (t-^) ) j | = 0. 

V* ilA(t,)a. e -bit t )IU 2 ja e - 2 A'tt,) bet,) 

a e = C Act,) A ct,))' J A (t,) 

23 

But, Penrose in established the following equalities; 

. (i) A A + A = A 

(ii) (A A + ) * — A A + 

Therefore 

from (i) A'ltjl Alt,) A^(*t,)) ft,) 
from (ii) A It,) A (t,) A^Vt,) = A (t, ) , 

The pseudoinverse is also known by the name generalized 
inverse. 
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then A- r t-t. i = 

whoro A 1 ( I | ) A { t ^ ) is m x xn . 

If A " 1 Uj) exists (i.e.: rank Att^) » m) then the mini- 

mizing a e is defined by: 

a e = A + (*b x ) ’bet,). 

The parameter estimation problem will now turn to 
modeling techniques. Unfortunately, modeling techniques 
-also live in the shadow of dimensional requirements. 
Therefore, the methods to be developed will be applicable 
only when certain restrictions upon the state and parameter 
vectors have been satisfied. 

3-3. Modeling and Implicit Estimation : 

Consider a system model of the form 

x e (t> = -f c;c e (t>,Utt; # a e <t)) 

- f (3 -id 

gi ven , tj e. Cto,!*! 

(superscript denotes actual system variables) 

where u is known from measurements (thus its functional 
dependence in (3-11) is omitted) , and a e is arbitrary, 
a e s OC . Assume (3-11) is completely controllable and 
observable. Also, it will be demanded that x (t) , 
x e {t), and a e (t) are monitorable over te[t Q ,T]. This is 
reasonable since (3-11) is to be synthesized (example: 

RLC circuit or computer program) , and the monitorability 
can be built into the model. The model will share whatever 
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continuity properties (2-1) possesses. The model, which 
is a parameterized version of the plant, will be used to 
estimate the actual plant parameter vector through either 
(1) numerical methods or (2) gradient techniques. 

The computational techniques to be developed will be 
philosophically different than those methods required by 
(3-9) . While the solution of (3-9) involves the general 
solution of a linear system of equations (i.e.: (3-10)), 

the modeling technique will use iterative routines. 

Formally, a model will be subject to the same con- 
trol effort as that acting on the plant. The output of 
the model will be compared to the output of the plant in 
some a priori manner and an estimate of the plant parameter 
vector will then be made. The model will be updated, and 
the process repeated again using the original control. 

The process of iteration will be continued until terminated 
by some decision device. 
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Figure 3-5 
Formal System Model 

Before the computational devices can be properly 

introduced, some important mapping requirements must be 

analyzed. It is desirable to insure that the parameter 

vectors achieved by either method (1) or (2) are unique. 

The Implicit Function Theorem will satisfy this goal. 

This well-known theorem establishes ■ some very useful 

properties of a system of the form F(x,ct) = z for a 

mapping F that is continuously differentiable with respect 

24 

to a. A theorem to be found in states that F is 
continuously differentiable with respect to a if and 
only if [3F^/3ctj] exists, and is continuous for all i 
and j . 








67 


13 

Theorem 3-2 : (Implicit Function Theorem) 

Let z, a F, g be m dimensional vectors, and x an 
n dimensional vector. Let F(x, a) be continuous for 
(x,a) near (x,a) and have a continuous partial derivative 
with respect to the components of a. Let the Jacobian 
with respect to a be nonsingular (i.e.: det [8F-®/9ou] 5 ^ 0 ) 

at (x,a) . Let z" = F(x,a). Then there exists e>0 and 
«5>0 such that if z and a are fixed, [ |z-z| J < 6 and 
| [x-x|'|<6, the equation z = F(x,a) has a unique solution 
a = g(x,z) satisfying ]|a-c?||<e. Furthermore, g(x,z) is 
continuous for j | z-z | | <6 , j |x-x| j <6 and has continuous 
partial derivatives with respect to the components of z. 

It is apparent from Theorem 3-2 that F(x,a) is a 
restricted mapping (i.e.: E m " l " n E m ) . Therefore, if 

the Implicit Function Theorem and model (3-11) are used 
together, special attention must be given to all defined 
mappings. Specifically, F(x,a) will serve as the pre- 
viously noted state and model output comparator whose 
argument, in part, will be the model (3-11) . 

Assume for the moment that F is well defined (i.e.: 
F:E n+m - 5 - E m ) . How then may F(x,a) be constructed so as 
to yield a computable a? The mappings to be considered 

(i) F<x,a>= ( Kct;.) - ^(t- tl a)) 

(ii) Fcx,aw ( K(t-) - 


are : 
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for t € Ct 0 ,Tl 

K(^)6E S known . 

Several consequences to be found in these defined mappings 
are: 

r*l 

They are point functions . 

Also, the solution of F(x,a) = 0 involves 
dimensionality considerations. (See the 
Dimensionality Restrictions Section) 

The engineer may also use some of his in- 
tuitive powers to enrich the solution space 
of F (x, a) = 0 . 

For example, the case where m > n, which implies 
m > q, it may be possible to consider (m-.q) parameters 
to be essentially nominal over some interval of time. 

In such a case, one may choose to fix these (m - q) 
parameters equal to their nominal value over this inter- 
val. This would admit a F:E n+q ■+■ E q over this interval, 
and the uniqueness of the q parameters would be the result 
of a satisfied Implicit Function Theorem. The interval 


If F was defined to be a bounded linear operator taking 
a Banach (or Hilbert) space into a Banach (Hilbert) space, 
then Theorem 3-2 would require an investigation with 
respect to Frechet [Strong] differentials^ . This 
will be considered later in this chapter. 
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[t Q ,T] can be covered by a union of closed connected 
intervals. For each of these intervals ,(m - q) parameters 
would be chosen to be essentially constant if possible. 

Another notion of essentially constant parameter 
vectors can be associated with F(x,a). Instead of in- 
vestigating the parameter alone, consider 


F n rx,a)+ F x n cx } a)(x- x n ) + 


.n 


+ i^(x 5 ax a- a n ). 


for x and a sufficiently close to x n and a n . Consider 
now the invariance of F(x,a) to parameter variations. 
Define: 





Suppose (m-q) o£ the CjCo-j - a" ) are small, and further- 
more are small with respect to the remaining q, then these 
(m-q) terms may be ignored. That is, their contribution 
to F(x,a) is negligible. The parameters associated with 
these (m-q) terms may be, for convenience, fixed at their 
nominal value; thereby admitting a map F:E m+< ^ E^. 

Example : 

Consider the following 


K, ~ ioo a, x 2 
K 2 - ioo cl 2 x,“ a 3 *z 


Fcx,a ) = 
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Also, require that and ) are restricted 

to some set about their nominal values . 

a , = i ±.\ 

OLz = I ± 4 
a 3 « i ±-i 

Here and would be the most likely candidates to be 
essentially nominal. Also, and K 2 are known, and 

“l\0 ^ K,6 -90 , K^ = -ioo 
- iioi.u - 3 oo .*$ , K” “ - 1004 

'£{a ,-<x?> = - f 00 * 2 ( 0 .,- a?) 

4 (a a" a 2 )= - loox^-a;) 

^ a 3" a 3 ) = - X Z '(CL 3 - (Zg )■ 

For this example, would be the most likely candidate to 
have its variational effect ignored. A technique may 
also be developed using a mixture of both of these methods. 
Another suggestion relates to the adaptive gain defined 
by (2-17) , namely G. The matrix G is a n+m x r and time 
varying. Partition G as follows: 



Let: 

Xj = lodb I 
X 2 = 1 ± -1 
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J.jC (m~q) voctors, say"^ 

a.iQ-i 


y. t 


3ri a i 


j t = l-i 2» 


m 


are negligible, component by component, compared to the 
remaining q vectors, then the corresponding , of which 
there are (m-q) , may be assumed nominal. This is because 
they have a minimal influence in the adaptive control. 
Then a mapping F:E n+< ^ -*■ may be postulated. The pro- 
blem of constructing a mapping F by intuitive means, 
rather than by technique’s discussed in the Dimensional • 
Restriction section, relies on ad hoc methods and the 
engineer must think seriously before ignoring totally 
the effects of (m-q) plant parameters. Assume that 
F(x,a) is now the well-defined map F:E n+< ^ •+ E g . For 
simplicity, let q = m, remembering the original maps F ' 
may have been defined for q ^ m. 


3-4 . Numerical Techniques 

The numerical device to be investigated is the 

Newton Raphson method. Alternative methods posed' by 

Todd and Ward, and Rich and Shaw, are also applicable 

21 

to this section 

'Consider F(x,a) = (x(t^) -<}> {t^? a e ) ) 
where: - - 

The vector x(t^) is a point in the state 
output space at t ~ t^ , t^£[t Q ,T]. 
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<f>(t^,a e ) is the solution of the model 

equation (3-11) at t - t., x e (t.) = x(t.) 

■^*3 3 

given, t^ < t^ , and a e given. 

F (x, a) has the following properties: 

(i) F(x,oi) - 0 

. 3 

(ii) F(x,a)sC [t Q ,T] implies F a (x,a) is 
continuous on | Ixtt^) - x e (t^) , 

a - ct e | | <. s . 

Now, if F (x,a) is nonsingular, then there exists a unique 

Oi 

0 0 

a such that a = g(x(t^) , x { ) ) in a neighborhood of 
x(t i ) , a(t^) . To compute a, let a = a e ^°^+g, where a e ^°^ 
is an initial guess of a. By Newton Raphson's method 

g <vo - u s <t 1 ),a e ' 01 t g lk) ]"' 

[F« e (t i ),a eco hg (k) J 

with = 0. At each iteration step k a g k is computed, 

. 0 (o ) — * (ki) 

and if a is sufficiently close to a, then g g 

always as k ■* 00 . 

For example, suppose the plant is defined to be 

<i) Xtt)- *f (X£t),oL(-t) ->a } 

_ a x(i) + u(t) , 

X f o ) - O, 
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and CL ■ = 1 

CU-t ) 5 I 


o v e r 


tc [o,i] 


then X(/)= I - exp (-!) 


Consider the model 

at) o- e x e (« + utti 

X e ro) = X CO) 

& e arbitrary. 


The solution to (ii) is: 

C i - exp c-a e -t)> 


a e 


and 


P_ e Cx e £ ( ) , cx e )) = 


- ^[xct) + ( i/Q> e Xexp(- a e )- t )] 


a' 


a £ 


w ^ j 

[F a eCx1p,ao))] is nonsingular for 


a' 

all finite a and 


[E.eXx^n,^!))] "'= (a e f 


(-/tCi-t a e ) exp(-a e )) 


For g^°^ = 0 and the initial guess of a being 
a e(o) = l/e 


Cj a K o+-.45 -.45 ^ a eco = .95 
g <2) =.45+.OG95 =.' 51 95 CL^ 2 ^ 1.095 

g C3) = .5i95-.0i90 =.5005 a e<3; = Loo5 
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After three iterations , convergence becomes 
apparent. For g -»• 1/2 , for some k sufficiently 
large ; 

+ 1 K X _ 1 -expc-n -( i-exp(-i)) _x = 

2 M + 2 exp H)) 2 ^ 

which implies 

eco) (k. 4 . 1 ) _ 

a +<3 ~ a 

In this example, the model' 1 s trajectories were known 
in closed form. The quality was used strongly in the 
computation of a . If a closed form solution was not 
available, then the iterative method would have been 
based totally on numerical results. 

A special case of the technique just developed would 
allow for one iteration, then again sample the plant data, 
namely u(t) , x q , x(t) or x(t) for te [t^ , t^] e [t Q ,T] and 
continue. The data from the second sampling then would 
be influenced by the updated a e (achieved by one iteration 
of the last sampled data) in the adaptive control- loop. 

This process may have merit if significant convergence 
is made -through one iteration in that the adaptive con- 
trol would be operating with a "fresher" parameter 
estimate. However, one must realize that frequent sampling 
will reduce the possible number of iterations which may be 
imbedded into a fixed real time interval. 
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Termination of these routines will be introduced as 
follows : 

1. Completion of a fixed number of iterations, 
say N. This can be related to a fixed time 
interval over which iterating will be allowed. 

2. Satisfying the e>o condition, 

g‘ «< e 

p a positive integer 

3. Similar to 2, except to minimize the effect of 
possible oscillatory convergence of g, 
require that the variation 

. £ \\ 3 i *lq Jl ue 

je= i 

p a positive integer. 

4. Combinations of 1, 2, or 3. 

1 

The. termination device remains rather arbitrary. A 
few further comments on its structure will be offered as 
a guide to the designer. Suppose a mean (or maximum) 
estimate of the real time required by the computer to 
complete one iteration is At. Let AT denote the amount 
of real time that the computer will be allowed to operate 
on a specific set of plant data. Then H AT/At U (see 
Symbols, Chapter 1) represents the mean (or minimum) 
integer number of iterations processed for the specified 
plant data. To add an additional degree of design free- 
dom, consider AT to be either variable or fixed over 



76 


[t Q ,T]. For example, let AT be ' proportional to an a 
priori smoothness assumption on the monitored (or derived) 
plant data. 

For a particular t ^ , t ^ being the initiating time for 
a sampling interval, the basic structure of a typical 
"fixed" -sampling interval is given in figure 3-6.- 


COLLECT 
PLANT 
DATA AND 
INITIALIZE 
MODEL 


£ 


to- 


A t 


iterations 
— PERFORMED 

! . r 

5TART ITERATIONS 

AT~ 


PATA PRocEiSiNq /MteavaL 

Figure 3-6 

-Fixed- Iteration Interval 


AH 




The iteration devices proposed by methods 2 and 3 (i.e.: 
satisfy an e>o condition) may be though of as a "rate of 
change" test, in that 


II s L ii .• H S - 9 u 


ti+-n c 


l(i+i> -U 


The designer must establish a workable trade off between 
too large an e, which might sacrifice convergence in- 
formation ; and too small an e , which may never be achieved 
in a reasonable amount of time. 
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Figure 3-7 

Plant - Model Diagram for Example 


Earlier, the notion of acknowledging the existence 
of a weak time dependent component of the parameter 
vector a was established. (example: drift, aging) The 

parameter estimators thus far developed were point function 
estimators, which implies that the estimated parameters 
were constants. An approximation of the time varying 
nature of the parameter vector can be accomplished with 
generalized step functions. 















78 


Definition : A generalized step function is one in which 

a measurable function takes on only a finite 
number of different real values, (i.e.: 
simple functions) . 

The claim is that for some sequence of generalized step 
functions 



2 6 

Proof: Halmes 

This is to say that a generalized step function with 
a finite collection of values can be found which is 
arbitrarily close to a(t) . 

Consider for example: 

,d<t )*t: 

Define: 

(, vf It! ^ n 

o i-f lti>n 
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Figure 3-8 

A Generalized Step. Function 


Therefore, one is motivated to find a "good" approx- 
imation of a(t) with some refinement of the sampling 
interval . 


Define: The kth parameter estimation to be that para- 

meter vector based upon the kth set of in- 
dependent plant data, k = 1, . s, s finite. 
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Definition : The plant data is said to be independent if 

for all A , k=l, . .., s 

rf 

Define: The kth iteration interval (see figure 3-6) 

for the kth parameter estimation to be 
[t^, AT +t^] , remembering AT may be fixed 
or variable depending on the termination 
strategy employed* 

Define: The kth data processing interval (see figure 

3-6) for the kth parameter estimation to be 

P k . 

P fc - tt>, tSSATl 


The data. processing scheme proceeds as follows: 

1. If operating with only one complete model 

(with memory) in the loop (see figure 3-5) , 
perform operation P 1 for some-t^ >_ t Q . 
Follow P 1 with P 2 ... P s , p k OP i = 0 for 
k, A — 1, . . . , s, k^A and for the fixed 
terminal time T, [t? , t'jflT = for 
t'e(t?, t? + AT S ) , but AT? Pi T need not 
be empty. The spacing of these intervals 
is a matter of the designer's judgement. 

For example, one may choose to initiate 
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the data sampling to begin at t^ = t Q 

■ x 

or at some delayed time t^ > t Q . Also, 

V- 

one may choose to follow P immediately 
k+1 

with P , k+1 1 s, or delay it in time. 
The greater the expected rate of para- 
meter change, the more densely packed 
should be the P's. Obviously consider- 
ing a P s+1 , such that jtf + + AT 2 * * S *JfiT=T 



Figure 3-9 

Example Sampling Interval 

2. If operating with several complete models 
(with memory) in the loop (see figure 3-5) 
follow the outline of events established 

in (1) for each model. However, require 
that the intersections of small neighbor- 

hoods about the t^ associated with each 


model be empty . 
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- *-*,* ] 
- J 


Figure 3-10 
Parallel Models 

Consider a second example 

F (X., a ) * c hto - -f c x e (t t ) v a e ;) 

where X € E and *f CX e , <X e ) is given by (3-11). 

F(x,a) again satisfies the following 

(i) Fcx(t i ),d)=o 

(ii) F e (X ^ } CL e C± l ) ) is continuous on 

Va* 

ll xit L )- X e it L ) , a - a e ct j II < €. 

and if F^e (x,a) is nonsingular, i'hen there exists a unique 
0 

a such that 

a e = <3 c x^cti >, x (-t-,)). 

The desired a is again obtained from Newton Raphson's 
method. It should be noted that x(t^) may not be monitor- 
able and may have to be computed numerically. 
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Consider the numerical example: 

xtt>= -fcxi-t), CLc-t) 3 d) 

- dxli)+ UCt) 

and _ 

X c-fc 0 )= o 

a s i J t£ [t Q j] 

act)= i j 

Then ■ 

^CD-exp (-D. 

Define the plant model to be 

X e (t)= GL e X e Ct)+ UCt) 


for arbitrary a e 

and x e (t Q ) = 0 

a e ) = 

C> <Srt,)-a e x*(t.).-i ) . _x e (t > 

6 a e 

Again, if a e ^ • = 

= 1/2, g (o) = 0 

<3 (,) =.)43 

a eo) = . G43 

9 C2> = .285 

=s> a eC2) = -7 25 

9 C3) = .280 

a e<3) *.78 o 

for g (k > = l/2,g< k+1 > = ,/ 2 = g' k U .« 1 !k > - 1. 


The closed form solution of F(x,a) was again available/ 
which will not always be the case. Therefore/ F e(x e ,ct e ) 
as well as f(x e (t^),a e ) may require numerical computations. 
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The truncation devices which are applicable to this pro- 
blem; as well as data processing intervals, have been pre- 
viously considered. (See figure 3-11) 

It may be noted that the rate of convergence of 
numerical example one was superior to that of numerical 
example two. This is not to be considered a general rule. 
The rate of convergence is a property of the problem 
(i.e.: F(x,a)) considered. Another computational device 

which has parameter estimation applications is the method 



Figure 3-11 

Plant - Model Diagram-. for Example 
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3-5 . Gradient Techniques 

The particular problems to be investigated are: 

1. The minimization of a function on E^. 

2. The minimization of a function on E^ with 
side constraints. 

3. The' minimization of functionals. 

4. The minimization of functionals with side 

constraints. . . 

These methods will vary in their utility from problem to 
problem. In some instances , approximations will be re- 
quired. In all cases, the required function, or functional, 
will be assumed to be weil defined. 

The first method answers the following question: 

Find a local minimum of a real valued 'function 

defined on E g . Let z = (z,,...,z ) be a point 

r q 

in some region T* 5 E^ . Let C(z) be a real 
valued, continuously differentiable function 
defined on T* • Pick a z q , and consider all 
curves z(s) parameterized by arc length s, 

Z: jdzns>/</S l = lu'fSMc * unit 

velocity. For such curves passing through 
z o , find a curve which minimizes C(z(s)) as 
rapidly as possible. 
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■ Figure 3-12 
Minimization Process 


The solution is found thusly: 

4Cczts>) =V Cczcsj) - z'cs> 

d s Z 

choose 

- ZVS; " - V Z C<Z(3)) ? ZCo) =Zo 

\V z Cczcs»\ 

f ° r Cczcs)) -/ o. Therefore dC<Z(S))/ds 

- - i V 2 C C ZCSJ ) | . This defines a curve which 
is always normal to the level lines of C(z(s)) as implied 
by figure 3-14. If C(z) has a local minimum,, then the 
scheme proceeds to it with unit velocity. In application 
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one may choose the following: 

z'<:5> = -k V Z C (zcsy ) , zco =z Q 

where ‘k adjusts the rate of convergence. If k is too 
large, then the minimizing trajectory may pass by the 
local minimum with sufficient velocity so as to tend 
towards another possible local minimum. If k is too 
small then the convergence rate is needlessly too slow. 
Example : 

Consider a plant 

X = CtX + U, XCO) = 0 
and CX - 1 ) Cl s I over *£ ^ £.&•> \ 3 „ 

Therefore 

Xtt) s I - exp C-t). 

Consider the model 

x e = a e x s + u » x'co.--o. 

Then X e {*> =( l-expf-a e t ')}/ a e and let 2 C ( CL e > = 

( ( i } Ct e ) — x < t ) l 2- ' where <£> (± f Q?) is the solution 

. . . 0 
to the -model equation with arbitrary a . 

Then £Cl a € ) * c'^£l,g e )-XCl)> - , ,Xfl) monitored.- 

5a e £ a e 

Note : In this particular example, the above partial de- 
rivative can be directly computed. But if of) was not 

in closed form, then the partial derivative would have to 



88 


be approximated by the derived partial 

s? <&ct } a & ) 

(a e ~a e ) 

for a e sufficiently close to a e , Another technique of 

determining this required partial derivative is given by 

27 . 

Margolis ■ . He considered the ordinary differential 

equation . 

(i) Xf*) + QX(t)* ft ft) , X(t a ) =X 0 


which can be ^^?ritten trivially as 

(ii) dxct) •*> ax(t)=/?(t) , ie : a>K(±) = a X.(t) 

at at at 

Now take the partial of (ii) with respect to a and note 
g (t) ^ f (a, t) . 

ft N 

(iii) X(t) + a oxct) fX(t) =o 

a a at a a 

Let 17 (t) r (t) , then (iii) becomes 

Sa 

(iv) 

_ a i7tt> - -xct) 


where 

Utt) K 


the solution of 

axtt> 

aa 


(iv) , 


for x(t) monitorable is 


if a is constant. 
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However, if a is time varying, then 3x/3t = 3x/3t + 

+ ( 3x/3a) (3a/3t) . So the validity of (iv) is predicated 
on a beinq either constant or slowly varying in -time. 

If this be the case, then the solution to (iv) is a good 
approximation of 9x(.t)/3a. 

Therefore : 

c>Cc a e ) - ( ( i- exp(-a e >-x(i) 

3a e V 

-k OCca e c5>) t -for a 

b a e cs) 

and JPi’m CL CS) Q. and if G,foj=GL } GLCS) = CL (Q, e <s i) =O m 

S->oo 

The solution of the above equation is nonlinear, and can 
be solved numerically. 

Example: 

g 

Consider Z C<d) =(k c <i } a e >-xn)). . 

Then, with computations similar to those found in the 
previous example 

<a e cs>)'= - k ( l -exp C-a e <s>)- x(n)exp C- clcs)), 
CLfo)*CL 0 X(l> monitored. 


f- 


-(i+-a c )exp(- 


a € )^ 



H 


Again lim a (s) ■* a, and for a (o) = a, 

S -)-QO 

a^s)=a ^>Ca e cs)/- o. 

A solution . diagram of the previous example is given in 


Figure 3-15. 
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Method 2 has the following formulation: 

Let C(a) be given and C (a) ec 1 [t ,T] . Consider 
the minimization of C ta) subject to r<.m constraints 
g^(a) = 0, i = 1, . r , where g^ (a) eC l [t Q ,T] . 
Consider all curves along a(cr) , with a(o) = 
which satisfies V a g^ Ca(cr) ) • a 1 (cr) = 0 (see figure 
3-14 X 



Figure 3-14 

Side Constraint Problem 
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Among those a(o) , find the one which minimizes 

2 8 

d({a(b))/da. It can be shown that the' 

solution to this problem is : 

r 

a'c cr)= -v Cca(cr})+^ ^^<3k ccu<:n) 

a k=) 

where X’ = (X^, . X^) , X =’G ''"y if G is 

nonsingular, 


(G is a Grammin matrix) 


and 


y= 


y, 

* 

* 


’ yi =( ^agi (a >>^ Cccu) 


However, if one investigates the possible choices 
0 0 

for C ( a ) or g ( a ) , namely : 

(a) c x e (t l ,a e )-xCti)) p =o 


(b) (x e (t- , a e >- = o 


for p, q positive integers, one would soon 

find that a’ (a) = 0 if x e (t^), x(t^) , 

x^t.) are monitored exactly, and x(t.) is 
l l 

monitored or computed exactly. If' a* (a) = 0 
then a (a) = a(o). This situation is a result 
of g(a ) = 0 being equivalent to C(a ) = 0. 
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In other words, (a) or (b) used as a side 

condition will only introduce redundant 

information into the parameter estimator. 

Then the minimization process can only 

0 

yield a a(c) which equals the fixed a 
and thus a'(a).= 0. Therefore, this 

method is of little use without some 
modifications. The modifications to be 
considered will relax the side con- 
straints. This will serve to allow 
uncertainties in the monitoring or 
.computing of x e (t^), x(t^} , and x e (t^). 

Also, the relaxation will suppress the 
equivalence relation between g.(a ) =0 
and C(a e ) ■ This condition will be arti- 
ficially introduced with "Valentine's 
29 

Device" . Valentine's claim was that 
inequality constraints can be reduced to 
equality constraints by increasing the 
dimension of the constraint space. 

Suppose a e 1 = ( a e , . .., a^) • and there are Jl<m con- 

straints ( (X e ) ^ o given . 


Define : 


,a“ ,af ^ Y J 


-2 


“ 9i h af.-O'McaL/] 


j sl t , 
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.a 


.m+Jl 


( OL } = in t. subject to 

constraints g . (a ) - 0. Notice that g.(a )=0=> g . ( a ) <.0 . 
3 3 3 

A Ag 0 A A g 

Then, for C(a ) = CCa ) , the minimum of C(a ) is the 


• 0 0 . 

■minimum of C(a ). In particular, each of the (a ) shall 

be expressed as (ct where represents an a priori 

error estimate associated with the j constraint. Thus, 

associates a number with, in a sense, an allowable 

error tolerance resulting from inaccurate measurements or 

computations. For example, let g.(a ) = (x.(t^) - ct^x (t^) 
2 

-1) s.e . . If was to equal zero, which would be used if 
3 1 


perfect system information' was available, then tt ^ - 

C Xj C*fc i ^ — i ) /x e (t L ) 

certainty, 


€.0 - 
a.f>o 


For £^>0, implying system un- 


yfe- /x € <l t ) t ( x. (t- L ) - 1 )/ x e ct £ ) 


£.tO 

af 


If the system is slowly evolving in time, the tech- 
niques which determine the minimization of a function at 
a point have been shown applicable. Data can be collected 
and processed over some interval of time, say . If 'the 
parameters do not change significantly, then the generated 
parameter vector at the end of this £ interval is a 
good approximation of a for some future time. 

One may desire to base the estimate of a based on 
more information than used for point estimation. For 
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example, let a e e In other words, one wishes 

to investigate oe e («) rather that a e (t^) . This is accom- 
plished with a functional format placed on the problem. 
There are several dissimilarities existing between a 
steepest descent problem, using a functional structure,- 
and a structure. First of all, the functional problem 
will be worked in Hilbert space, denote H. Secondly, the 

notion of a directly computed gradient no longer exists. 

■ • . 25 

Instead, one must consider Frechet (strong) differentials 

However, the problem will be simplified by requiring that 

25 

the weak differential exists and equals the strong . The 
existence of a strong differential implies the existence 
of the weak differential, but the converse is not nec- 

30 

essarily true. Then by the Riesz Representation Theorem , 
a gradient operator may be defined. 

Definition : (gradient operator) 

For a Hilbert space H,- and a continuous .linear- 

I p , 

functional C (ct Q ) on H-, there exists a unique 

element \7 eC(a e ) (the- gradient of C(a e ) at 
o 

in H) such that: 

C'laf )h Gal), h) 


* 

Definition: dk, 2 ^ i- s set °f all measurable 

functions that are square integrab-le 
over [a] . 
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where {,) is the Hilbert space inner product and 

c'«x*)h » Sim C(.a% +Xh ) -C (ctS ) 

X-o X 

for a e and h in H, X a scaler, 
o 

The functional minimization problems are: 

1. Minimize a given C(a e ) for a e eH with no side 
'constraints . 

Solution: Path of steepest descent. 

(o e «r))'.-V t c C(aV». 

2. Minimize a given Cta e ) for c* e eH arid constraints 

g. (a e )= o , isij k. 

Solution: Path of steepest descent. 

A 

(a € c cn)'^ -\^ e (Cca e fcT))+2^X l < 7g l ca e ca-}), 

i-i 

X^ previously defined. 

The construction of C(a e ) must demand that the minimization 
of C(a ) would imply a a over some interval of time. 

For example: 

a- € S 2 

C< a e )-j * (xit)-cj>( t , a e >, x it) - <pc±, a e )) dt. 



97 


Example : 

This problem is motivated by the linear regulator 
problem. The problem will be offered primarily as a 
device to exhibit the techniques required of (1) or (2). 
Consider the special plant and the computations paralleling 
a technique used in the solution of the linear regulator 
problem, with appropriate modifications'. 

Let the linear plant be defined to be: 

XHrwClt)d(t>+ 

X (tj)=X 0 . 

where f(t), A.(.t) , C('t) are known on te[t. ,t. ] . 

This particular- plant is of the form x = Ax + Bu except- 

for the C(t)ct(t)' term. The ' term f(t) will play the role 

of B(t)u(t) and C(t)a(t) will serve as a biasing term for- 

1 

the plant's input forcing function. One may consider f(t) 
to be the external forcing function, and C(t)a(t) to be an 
internal forcing function 



Figure 3-15 
Example Problem Plant 
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Let the model be: 

(i) . 

x e c-t>.= C(±)a e ct> f A(tu e (t) + 
a e eX 2 Ctj,to 

X e ltj)=X 0 

Then , for xnt>=x f given , 

Ul) X e £t i ^X<t i ,t i )j( e C-t j ) + 

+ f [CcT)a e CTj 4 --fcT)]dr 

*j 

where Xct ,tj) is the fundamental matrix for 
the.. homogeneous part of (i) . 

The cost functional to be minimized will force the para- 
meters to vary only in a neighborhood of their nominal 
values . Let • i 

(iii) ! t- 

C ca e )= f * c a g ct>-a n , a^-a" ) A-k 

• J±: 2 

j 

± 11 a g - a n ii . 

* - 2 

Also, the following constraints will be imposed: 

( iv) ’ 

•0^(a e )= C \p,a e )-<r K = o 
k-- 1, — >s 

for x(t) e E q , where: 
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(v) 




tp (t) is the kth row of ijj(t) 


and 


J,r 

(vi) 0 is the kth element of 

.t : 


-X (tj - f Xct- ,L)*fcT)<dT. 

T* • 


Therefore ( a. ) becomes: 


/ <X(t i 4 T> CcT) a e cT) -xtt t )+ ))-dr + 

t-t 


t- 

' L 


Xu^ri-Fc t) dr =o 


‘j 

This implies the model will be required to satisfy the 
terminal state conditions of the actual plant. One might 
also choose to use Valentine's method to represent some 
allowable terminal error. 

Let DC ca e , h ) « C S7 a C c a e ), h ) 


= C c a 6 -*- X h 

dX 


X = o 


- % f 'dc a^Xh -a n f Ar 
' 2 - 4 , dX 


= f Va'tXh -a n )hdT 


= Ca e - a", h ) ^eCca c ; =(at a";, 


X=o 


X- o 


V a ca e >= </> 
a J k 


similarly 



.10 0 


(vii) G- Ucg,a)] ,M- 

where : 

3kx = < V a 3k (a - s> ''¥Qt <ae) ')* ( 

then 

CitjCcti) K cti.tjj] 

X a £ ,t.j > = X t* £ ,tp MXct L ,tp 

• where M is a controllability matrix, there 

[ + 1 

fore a nonsingular 


The path. of steepest descent becomes the solution to 


(viii) 

d a € co 
d€ 


= - V ad^EK v ^< (aS] 

=-( Qe :a >+£ 


A necessary and sufficient condition that (i) be con- 
trollable (given originally by Kalman, and to be found 
on pages 187-188 of 28) is that: 

M ■= / X CxiCmCiX) ( X"(T)) dt 

Ji ‘ -i 

is nonsingular. In particular, if X ' (t) C{t) C ' (t) (X (t))' 

is nonsingular at just ona--t, then.M is nonsingular for 

all t. 
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where 

X k 5- C G 'y ) fc 

a K *(Va*C,v % ) 

= a", ft 11 ) 

„ <- a k , i /'* 5 

- cr k -^ Jrt^tiGTiaJcIf 

= °- k -K< 

Therefore (viii) -becomes 

- - [a e ce,t>-a n ] X rt j )G'p r +Na ri ] , 

d € 

or 

£ 

a e (€,-t) -e>xp(-€)a e co,-t.j)+ expc-^^^xpcrj LCc-t) 

[X”it}UX it L )Q~ J (cr + Na n )+^ n 3 d? ^ 

- expc^e ) a^cojt) + C i-£*p c-€ )l Ec'inE^ &)] 

X&i) q _, tcr + Na n )+a n l 

The a e which satisfies the minimization of C(a e ) with the 
given side "constraints is found by letting e-^oo. Also 
note that because exp {-eJ-^O, the final a is independent 
of a e (o,t)’. 
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3-6 . Non-determjnistic Parameter Estimation 

So far the various parameter estimation devices have 
had a deterministic structure. There has been some active 
research associated with the problem of estimating the 
"best" parameter with respect to some given cost index 
with sufficient a priori statistics known, or assumed known. 
This research paper will not attempt to enter a detailed 
analysis ‘of non-determinis tic problems- The techniques 
involved in the stochastic problem are manifold. A useful 
device, which will find the parameter vector, minimizing 
a cost index of the form | jx(t)' - x e (t) | | , is an extension 
of a scalar problem solved by Aoki . It essentially 
uses the property of a sufficient statistic, a differential 
difference representation of the plant, a normally dis- 
tributed parameter vector with unknown (but constant) mean, 
and a Baysian decision rule to update the estimate of the 
parameters. The parameter estimation is accomplished by 
minimizing .the expected cost, starting with the last stage, 
then working backwards in time, and using previous ob- 
servation to update the estimate of the parameters. 

Although no definite parameter estimation procedure 
has been established, it shall be assumed that the tech- 
niques offered are sufficiently rich in number and variety 
so as to produce a "good" parameter estimate, if one does 


exist. 
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then working backwards in time, and using previous ob- 
servation to update the estimate of the parameters. 

Although no definite parameter estimation procedure 
has been established, it shall be assumed that the tech- 
niques offered are sufficiently rich in number and variety 
so as to produce- a "good" parameter estimate, if one does 
exist. 
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CHAPTER IV 
ERROR ANALYSIS 

4.1' Approximating Parameters 

Consider again the vector differential equation 


~ Ay' 


■ A 

! B - 

_ J 


-Ay' 

-APj 

X 

_ C 

i -Ay 


-AP- 


with its 2 (n+m) boundary conditions given. The partitioned 
Ay vector was defined to be JjlxjA'aJ', where Aa = a - a 11 . 
Ideally, the estimated a should equal the actual 

: k 

parameter vector a . (note: In Chapter III the actual 

parameter vector was denoted a) ; Suppose a has a value 

* . - • 
other than a . Such an estimation error would give rise to 

two solutions of the Riccati equation (2-15) corresponding 

e A e n * a * n t 

to Aa ~ a - a and Aa = a - a . Nothing has been 

said at this point about the two solutions being dissimilar. 

The system parameters by assumption belong to some 

differentiable class of functions which implies some degree 

of smoothness. However, the class of functions which the 

estimated parameters belong to has been left to the designer 

to choose. This can be exemplified by the case where the 

parameter vector is a composition of constant vectors over 

disjoint intervals. That is, a piecewise constant vector. 

- 0 * 

One would desire that a -a , but this would necessitate 
0 

a belongs to some smooth class of f unctions . The ex- 
hibited computational methods would, however, forbid in 
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general the realization of a in a smooth manner. As a 
result, a e will be necessity be imhedded into a larger 
class of functions. For example, simple functions or 
piecewise continuous functions. This "large" class of 
functions would allow the designer a high degree of flexi- 
bility in choosing which parameter estimating device may 
be used. The choice of measurable vector valued func- 
tions will serve as the "large" class of functions to be 
considered. This class will be denoted 


M [t 0 1 tJ 

such that 

a e e M [Lj] 

M measurable on [to,r] }. 

This class of funtions is very rich in estimated parameter 


vector candidates. For example: 


e 

a tt) = 


t 3 /6 


a t € [o,: 


or 


t/2-'/3 , t€( 1,2 

I , -fc€ Co, 10 
o C * ■» 2 1 

, let CL e €p[o,l] cn M [o , l] for example , and 


then dVfo 

d t 3 


n 


a 

*• 

a 


= 1 

^ j +*t 



or, let CL% P jj) , ljo [0,1} for example, and 

a e = j I , t€[o,«/ 2 ) ) 

I I */{2 * t » */£ ( 

[2 , t € 11 



Figure 4-la 
PARAMETER PLOT 


Then form 


A a e = a € - a n 
A a*- a*- a n 


one achieves 



Figure 4 -lb 


A PARAMETER PLOT 
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Because Aa? is an m dimensional' subspace of 
[Ay |ApJ" ^ E 2 {n+n) , the righthand side of (2-15) is allowed 
to be discontinuous. One is interested in how this dis- 
continuity affects the solution of (2-15) . With a c or a 
defining Aa and a 11 fixed, it is possible to determine the 
variational bounds on the solutions of (2-15) for varia- 
tions in Aa using calculus in the sense of Lebesque. But 
to generate insight into the problem, the study of dis- 
continuous vector fields will be pursued. 

4 . 2 Discontinuous Vector Fields 

To motivate such arguments one can study a common 
application of discontinuous vector fields in control 
theory using closed loop state feedback control belonging 
to some constrained set. Problems of this class involve 

such notions as solutions in the sense of Filippov 

32 

and stability with respect to measure . H. Hermes dis- 
cussed such concepts and developed some additional exten- 
sions. He chose to consider a control system of the form 
x = ' X (x) 

• ,-n r-*" 

It: X = cjtx, UUxn, X € t- 9 t - (4-1) 

It should be noted that this forms a special class of 
problems in' that the control effort is a function of the 
states of the system only. The control u(x) is chosen 
from some control set ft and the target set consists of a 
manifold ir in C0,“)X E n . If g is bounded and Lipschitzian 
in both arguments and u is a given Lipschitzian control. 
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then (4-1) has a unique solution for x(0) = x q given. De- 
fine the solution of (4-1) to be <j>(t,0 / x Q ), -where the 
three arguments of <j> are to be read as the solution at 
time t initiating at time t equal to zero with initial 
data x q -. A solution which reaches the target set at 
t = t^ will be , represented as <}> (t^ , 0 ,x Q ) em. A reasonable 
question to ask is: If the target set tt has a dimension 

less than n in E n+1 , can one ever expect to find a collec- 
tion of x ' s belonging to some' nontrivial neighborhood 
of the initial data x^, say "fj (x q ) d E n , and a time, say 
t.(x) , 0 < t(x) < °°,’such that the .solution cj> (t(x) ;0 ,x) ) eir? 
This is to say, can one show a solution starting from 
some n dimensional manifold reaches- the target set ir in 
some time t(x) and has a dimension less than n. The 

i. 

answer intuitively seems to be no. In fact, it is no. 

The reason- that the answer is false is that u(x) is 
Lipschitzian . If, however-, the Lipschitz- condition on 
u(x) was removed and u(x) allowed to be discontinuous, 
the answer may be affirmative. For ‘example: 

f 

Suppose u is Lipschitzian, then consider (4-1) in ■ 

„2+l , 

E , and 

TI = £x 2 ,x 1 = constant, t = constant J 
Then for no t(x) there exists cj> { t (x) , 0 ,x) ) £ J[ for 
xe Tj (x q )‘. 
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•u Lipsehitzian 

If -u ;is allowed to be discontinuous, then the follow- 
ing could occur: , ' 
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To develop the concept of solutions of differential 
equations whose righthand sides are discontinuous solutions 
in the sense of Filippov will be investigated. 

4 . 3 Solutions in the Sense of Filippov 

Let X be a measurable function defined almost every- - 
where in some domain G c: E n . Define: 


K{X(x>}* fl f\ £5{X(Ufx,S)-N)} 

where 


co ' denotes convex hull 

U(x/6) denotes a closed 6. neighborhood of x 
N 'denotes an arbitrary set of E n 
denotes Lebesque measure 


Definition : An absolutely continuous vector valued func- 

tion <f> on £o r Q?J is called a solution in the sense of 
Filippov of x = X(x) if for almost all t, <j> (t) eKjX (<|> (t) ) } 
Here K (X (x ) ) , in a sense , determines how the deriva- 


tive x = X(x) behaves locally. For. example: 

Consider the following graph of trajectories leaving 

2 

a neighborhood of a point x, xeB . 




•Figure 4-4 


-VECTOR 'FIELD OWE 


As U(x,5) becomes small, (i.e., <5, -> 0,), K j X (x) } becomes 
x, or K { X(x) J — X (x) However, the' following extreme 

t 

case may occur. Consider the following graph of tra- 


jectories . 



VECTOR FIELD TWO 
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2+1 

Here K j X (x) [ becomes a wedge in E . This can be 
interpreted to mean the following. If a noisy measure- 
ment of x is taken, then one cannot be sure, even locally, 
that this measurement error will not admit a totally 
different trajectory than the one associated with the 
true x. Therefore, one may generate a whole family of 
different trajectories in some convex hull, even as 6^-0. 

If X is continuous, as a special case, K j X (x) } = X(x). 

Definition: If there is an absolutely continuous function 

4> for the real variable t which satisfies some initial 
data and <j>(t) = X(4>(t)) almost everywhere, we call cj> a 
classical solution. 

If is a Filippov solution of x = X(x) , x(0) = x q , 
then for any e,6>0 there exists a measurable function 
£ : [o,T^j "*■ E n with | j £ | | < <5 such that a classical 
solution' <J) exists on £o,t"| for the problem x = X(x+ <?(t) ) , 
x(0) = x , and satisfies j | <j> - 

Here 

H II s ess. sup | | £(t)l, [VO } 

= infjiVI J j £ (t) j < M almost everywhere on ^0 ,tJ | • 
So, if one can show (2-15) has a solution in the sense of 
Filippov, then there exists an allowably small measurable 
error <£(t) such that the resultant solution ( t ) differs 
from the classical solution <f>(t) by no more than e. 

Claim 


Equation (2-15) has a solution in the sense of Filippov. 


Proof : 
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'A Cj' 


'A 

6 1 

[ A s] 

(i) 

A p. 


C 

-A'_ 

M 


let D $ 

'A 

£> " 


[D. 1 

C 

-A y _ 


D* 


L_ 



^ £* — * 

whose elements d. : 

V_J. 

1! 

1/ 

• • * r 


Also, define 


Aa « Aa*= a*- a n 

Because (i) is a Riccati equation which has a unique solu- 

t 

tion which is expressable in terms of K(t) f (K(t) a con- 
tinuous bounded' linear operator) and establishes the 
relationship 

ApL-t)- K. (t)A y(i) . (see 2-16) 

- Ay may be described as follows: 


{ii) Ay* ErAy.) = D 4 CAy, KAy)=('A+5K)Ay * 

•Therefore, all that need be shown to prove the claim is 
that (ii) .has a solution i-n the sense of Filippov. 

Call the solution of (ii) <|>(t) . Notice from the continuity 
of tj) (t) on- £t Q ,Tjthat ip is a solution of (2-15) in the 
sense of Filippov. 


Therefore , 
function S’(t) on 
fying Ay - E (Ay+ 


for any e,5>0 there exists a measurable 
■J^t Q , tJ with || • g | | <6 such that a ip satis- 
£(t)) exists and furthermore it satisfies 
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| | <j> - i|>| J < e . Although an £“(t) has only been postulated, 
one may hope that there exists such an £(t) which can be 
used to represent the difference between a and a ; If 
one considers such a representation, and if <?(t) is small 
in norm, it shall be shown that the difference between the 

■k * n 

solutions of (ii) corresponding to ia = a = a and 

^ ^ n . 

Act - a - a is also small. Therefore, one's attention 

is directed toward relating the magnitude of the parameter 

estimation error to the errors found in the solution 

space (i.e., <j> - $) . This shall be accomplished through 

the study of "stability with respect to measure." 

4 . 4 Stability with Respect to Measure 

Definition : 32 A vector field E for' which a classical solu- 
tion 4> of z = E(z) exists with arbitrary initial data z Q , 
is said to be stable with respect to measure if given an 
£ > 0 and T> 0 finite, there exists a 6>0 such that whenever 
S is a measurable function on £t Q ,Tj with values in E n 
•and | | S | | <6, for which a corresponding solution of 

z = E(z+ £(t) )■ 


z(t ) = 
o o 


exists on te jt Q ,Tj , then - | | $ - |<e. 


The last ' definition is a canonical definition for 


this problem. By that it is meant that this definition 
would have evolved from a straight calculus attack on the 


problem. .However, this approach would probably not have 
developed fully the powerful property which is nested in 


the given definition. First of all, the whole, or only 
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pari. of !;ho vector z may be considered to represent 
measurement errors. Also, the measurement error is 
additive which will lend itself to intuitive arguments. 
Another important feature is that one is working with a 
given error bound placed on the normed difference of <f> and 
4>, namely e. This allows the designer to establish some 
maximum tolerable error on the solutions of (i) in some 
a priori fashion. 

Claim ‘ 

The vector field defined by is stable with respect 
to measure. 

Proof : 

Consider e>0 given and te ^ 

classical solution of (ii) . Let t|rbe a solution in the 
classical sense of 

(iii) ALj = D,fAy K( A ))■ 

A Lf (~t a } given 

°r Ay = (/\ + 5K)Ay + (A + BK) $ 

Let<$>(t) be the fundamental matrix of (ii) . Let 
4>{t,t Q ,y) and ^{t,t Q ,y) denote the solutions of (ii) and 
(iii) respectively at time t and arbitrary initial data 
y Q given at t . Then 
c £(t,± 0 ,y)= 

4>€-t,tJc j + f cf>(t,T)(AfT)+6(T)K(T))^CT)clr. 

J~to 


and 
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Therefore 

/•t 

= 1/ J 4><t,T)(ACT)+&CT>K<T)) <? (T)dl r fl* 

*o 

4 lT-t 0 .| II 4*(-t,r)fA(r )+ Be t> Kct>II ll<?crj II , 

• T « [*.»r]. 

Therefore choose 

? * 

lT-t c | II<$>(t,T)(ACT)+BCT) KiCT) j/ ^ (4-2) 

If a more delicate 6 was desired, then a closer exam- 
ination of li (A(T)tBcr; Kcr;)ll for a given plant need be 

performed. 

Example : (scalar) 

Let f(x,u;.cO = ax(t)+bu(t) 


and 


C ( lO - 




X 2 (i :) + U* (t> ) d~t 


Then 


0, 


r- n 

a 

0 


f) , « 0x2 

x Ct) j - ( b ) 

ci i r\ 


. t* 


o 

o 


D* 


-7 

L -y 5 ' 




-a 


n 


n 


(±) 


~X n (t) 


O 

o 


For Act = a* 


n 


a 


c, c a constant under perfect measurement. 


From (ii) : 
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A x (l) - a n Axtt)+ - Ct> j Z [k, t Ax(t) + K-i2. c ] 
( b r ') 2 k l ,') A xit>+ (xYt)- a*) 2 k, z )c 


Aa - o> 

Suppose Aa is subject to a measurement error such that 



— — 

Axt-t) 


Axi-t) 

A - 

Aa 

- 

c +■ &ct) 

. t 

* . . . 


L * 


The term c + £(t) represents (a - a n )+(a e ~ a ). Suppose 
a e equals a almost everywhere on £t ,tJ. Then the "'’bad" 
data points belong to a set of zero measure. Therefore, 

|| <f (t) f | = 0 and <S = 0*£-||cf>“i|>||=0. This result 
correlates with the fact that two differential equations, 
namely , 

(a) U) = 

A UMt 0 >= 

and 


<b) -£(&>,/) 

Ojlt 0 l=6d 0 - 

and v measurable, 

where f is bounded and Lips chi tzi an in the first argument 
and measurable in .the second, will have solutions equal 
everywhere if v differs from zero only on a set of zero 
measure. If £ (t) is some finite {in norm) measurement 
error, then a bound 6 can be calculated from (4-2) . In 
this example only parameter vector measurement and/or 
estimation errors were considered. The $ resulting from 
an a priori 6 will influence which of the possible choices 
of a parameter estimator may be selected. That is,. there 
will be a prescribed 5 precision required from the • 
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parameter estimator implemented. 

4 . 5 Cost Index Error 

What variations exist in the cost index due to 
measurement (i.e., approximation errors)? Consider a 
neighborhood of y n (t) sufficiently small so as to admit 
a truncated Taylor series representation of H(y,p,u) 
with negligible truncation terms. Let C(u) be again de- 
fined as 


C ( u ) = 1 C XCT) 5 T*m> + 


f T 

J L (x 9 u) 

■to 


dt 


where L is Lipschitz continuous. From being stable with 
respect to measure and e>0 such that if [ | f° (t) | | <6 , then 
| |<J> - |<e. The vector <f>(t) and ijf(t) are members of 

E (n+m) were p xe viously defined. By Minkowski's in- 

equality it is apparent that 


II c£(t)-l/nt)ll S H(<£, ,<P n l ill- (.'jf, (t),--- , 

, ^it)) II. 

Even with the postulated small measurement, or estima- 
tion errors, the adaptive control Au will have its original 
structure . 


AU.Hr)= Gtt) M(-fc)ApC-t5 

where G(t) and H(t) are defined by the matraices of (2-12) 

in an obvious manner. But by virtue of the fact that 

Ap(t) = K(t)Ay(t), Au(t) becomes 

Auit)= Hit)Kai] Ay(t). 

Because L is Lipschitzian, C lu,) - C ( U. z ) = 
-t<X,(T) J TX j( T»-i<X i , t T),TX 2 <T)>+^[L c x 1 ,u,)-LlX 2 ,U J )]dt 
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can be written as 

!lC(u,)- Ctu z )IK lT-tJ K fJx r A 2 ,Ut-u 2 il ■+ 

+ K HX t (-r)-X 2 (T)t\ 

where K is Lipschitz constant, u, corresponds to the control 

* 

strategy with a-^ = a and ^corresponds to the control 
2 * 

effort with - a r £ . 

By Minkowski's inequality 

IlCtu,) - Ciu 2 )ll < lT-t 0 l R [ HX.-xpi+llu.-upi] + 

. + K II X,(T)-A 2 tT)l> 

or 

. II Cut,) - Cm*) II « lT-t 0 iK [l +l/lT-t«,l+IIQ + HK.ll] « = «' 

f or € given and 6>0 sufficiently small. 

Thus, for the € ,<S conditions satisfied, the normed 
difference, in the cost indicies based upon the 4> and xj) 
solutions are nothing more than a scalar multiple of 
If maintaining,^"' below some maximal value was of paramount 
importance, then one would solve for the last equation for 
€ and this £ would then define a required 5 . 

The error bounds found in this chapter are not 
necessarily the sharpest available, but will always be sat- 
isfied. Therefore, the designer may look upon them as a 
maximal guide . 

4 . 6 Truncation Error 

Consider now (2-13) and (2-14) with regard to their 
solution with and without ignoring the contributions of 
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truncation errors . That is 

(i) 

Ay- [A ib']| 


A t 

+ 

’x,' 

A p 


O 


, A/- r\ x \ 


(ii) 


Ap-[C j-A'] 


A y ’ 

+ 

x 2 

.Ap _ 




, X2-'h+mxl 


ApCTi given 
for some previously defined A^, and A 2< The vectors 
and A 2 will be used to represent the lumped truncation 
errors suggested by (2-13) and (2-14). Furthermore, for 
A^ and A^ being small bounded truncation errors defined 
over (y,p,u)eN (N defined in Theorem 2-1) there exists an 


€ ^ and such that 


d. 

sup | 

fXj 

[O J 

1 

^ €,■ 

sup 1 

x 2 

j - £ 2 


-t € [fca/r] and -for all 
1 y ^ p, u.j € M 


Let fi(t,t Q ) be the 2 (n+m) x2.(n+m) fundamental matrix of 


(iii) 


*Ay“ 

Y A & \ 

Ay 

Ap 

\c -A') 

Ap. 


that is 



5 <n ctj-to) 

. A Lj l±o) 

A ptt) 

— « 


.A pCO 

u - 


where Ap Q is the unknown initial costate vector. Partition 
fi(t,t Q ) as 
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^ i ri Z2 {-6,t«»)/ 

Define T to be (T being the weighting positive semidefinite 


matrix in (2-2) , 



(rBJ'vn x tn+m> 


which results in 
ApCT)^T 

This condition requires that at the terminal time T the 
estimated parameter may belong to all of E m . (Remember, 
Ay actually belongs to some restricted subset of E n "*" m ) . 
Also 


A y(T)= (T,t) Ay(t^n i2 CT,-t>/!tpc-fc>-t 

+ ^ r cn„< T ,T)g.^ n n ct, -o [x 2 ] )d r 

and 

ApcT) =n 2l (T,t) Ayit>+ n 22 cnt)Apc±) + 

+ n 2 I cr,T> j) A] + n 22 CT ) T)[Xj)dr 


- T AljCT) 


After some algebra, Ap(t) becomes 

,-J. 


A ptt) - [n 22 c-r,t>- t [TH^t) - n 2( cr,-t)J Ayft;^ 


r T ^ -I 

/ ([n 2Z iT,T)-T n i2 <T,T)]<fn (l (T,T> -n 2i ( T ,T> X, 

-'-t L - 


{Xj)d- 
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But 

K<«»[n, 2 <T.t>-Tn„<T.*>] ' l [Tn, 1 <T,ir 1 -n 2 |( T,i)], 

f p 

kxt>»t. 

The matrix K(t) has been previously defined. Upon sub- 
stitution 

T 

A pit.)* K(t„) A y (t.) +J C KCT) X J)d T 

Therefore , 

A y It) = [n „ rt,t»)+n , z ct,t 0 ) K it.)] A y «•„) v 

+ [([n 1 .(t,T) J -n 12rt ,T) k «>[-£-■]* 2 n ra <t/r>[x*] 

Let Ay(t) and Ap(t) be the solutions of (iii) , Ay(t Q ) given, 
Ap (T) =Ap(T) given. Then, for Ap(t) = K(t)Ay(t), 

Ay tt)*[h „ rt.t.) +n ,2 ‘t.'t.J * rt »'>] A y . 

Therefore , 

JJ Ayctv Agtt)H«ll/* Kc*>4-n„tt,T:)] [~i] + 

+ a n i 2 ct,T> [X 2 ] ) d'c 

^ \x-t 0 \[im l2 ct,x) KcT)+n ll (t f Ti(f6 I +2«n ra (t S T)Jl€ f ] 

for t , T£ j^t^T j , 
or 

(iv) il A tj (-B - Atjt-t)i| M € 4 + N c 2 . 

Therefore, the computable error introduced in (i) by ig- 
noring the • truncation is always less than a weighted linear 
combination of the largest truncation error expected as y 
and p range over some restricted domain. This domain was 
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investigated in Chapter II. 

4 . 7 General Performance Error 

Equation (iv) may also be used to establish an error 

* 

bound on the cost function. The difference between C (u ) 
(with truncation error considered) and C(u } (ignoring 
truncation errors) can be established as follows. 

With <s ^ and defined previously, consider to be 

€ 3 < sup \ o u ce 3 )| : t € [i 0 l T ]and -for all 

(y , p, iai e N , gi ven 

b^i 2-12. 

Therefore, for G(t) of (2-17) represented as 

Q (t) = 

R ( t) is rxn 
S(t) is rxm, 

iia*(t)-a e tt>U4 ij RitiilJI x^(t)-x e it)l\ + U Si-t)\\lla* a e u+e 3 . 

Also, it can be noted that 

<x*(t),t x*cn>-<x e CT),T^ e cr)> 

2 Z 

= 2^ T u CXjcT)-xf CT>) ^ r l/X*CT)-X e O’)tt 

t=i 

where 

F = max T;; 

define k to be a constant such that 
JL.X*CT^~ X C CT') 1) ^ } 

Then k P |j X*(T>-X C CT>Ji ^ P il X*(T> - X^CT) il* 

The difference in the cost indicies C (u ) and C (u ) becomes 

K is a Lipschitz constant) 
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li GuV Ctu^ll * lT-t 0 i K Olx*t)-x C (-tjU + llRit) 
ii x *tt>- x e <t> n +- 1! Stt ) lilid*- a. e il+€ 3 ) + 

+ kf I1X*(T) -X C CT)\\ 

- i T -*t )K[( i + il /?<*>?! + AEL- )NX*-*V II Stull J|a-aKj 

iT-t 0 lK 

from (iv) 

^ lT-t 0 j K ( 1 + llRi-fciH +(kP/fT-t 0 i k)( M^, + N e 2 )+ 

*IJSit>liila*- a e i i + € 3 = Ack^j^^a^a^r ) 

* 0 

for a = a , 

A^,^. , € 3 ,cl*» d 12 , D = A (k, € , , € 2 9 € 3 }F) 

For a prescribed maximum allowable value of A (k ^ } I7 ) , 

say \ , j | C ( u ) — . C (u e ) [ J ^ A . Then k, e-, , e 2 / £3 may be 
chosen (not uniquely) in such a way as to satisfy the A 
constraint. Choosing a maximum acceptable terminal differ- 
ence (i.e., ||x*(T) - * e (T)||), thus fixing k, one can 
find the neighborhood N (of Theorem 2-1)' such that the 
e^, ^ 2 ' e 3 candidates are satisfied. 

Because of the general treatment given to error bounds 
in this chapter, the error bounds generated were, in some 
sense, maximal. Tighter bounds may be established for 
particular problems through a more detailed analysis of 
its structure 
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CHAPTER V 
SENSITIVITY 

5 . 1 Sensitivity Index 

The previously developed machinery will be used to 
show that under certain local . restrictions a reduction in 
the systems sensitivity to parameter variations will be 
accomplished. The question of sensitivity will be 
developed qualitatively. The sensitivity index chosen 
should satisfy the following intuitive ideas: 

1. The ideal adaptive control is one which equals 
the optimal control for a particular problem. If 
the ‘adaptive control for certain parameter varia- 
tions equals the optimal control over those same 
variations, then the system's cost is insensitive 
to those parameter variations. 

2. If the adaptive control does not produce a cost 

C(u), (C(u) defined by (2-2)) which equals the 

minimum of C(u) over a set of parameter varia- 
tions, then the system should be considered sensi- 
tive to those parameter variations. The greater 

the disparity between C(u) and the minimum 
* 

C(u) = C (u ), the more sensitive to parameter 

variations is the system's cost. 

A sensitivity index S Y chosen will compare the cost of 

operation of a system subject to parameter variations to 

* 

the optimal cost of operation C (u ) . Or more succinctly 

S Y = | C (u* ) - C (u Y ) | 


(5-1) 
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where the properties of C(u ) and C(u^) follow directly 
from (2-1) and (2-2). Although the construction of a 
sensitivity index is arbitrary, (5-1) shall be assumed to 
be the canonical sensitivity index. With this definition 
of S' and Theorem (2-4) in mind, the following is easily 
shown . 

5 . 2 Sensitivity Reduction 
Correlary (2-4) 

Under the conditions stated in Theorem 2-4 

s e < s n 

where : 

^ 0 m 
u , u , and u have been previously defined. 

Proof : 

Apply Theorem 2-4 to S e a nd S n . 

This result guarantees that under certain local 

, j 

conditions given in Theorem 2-4, a system operating with 

an adaptive control strategy is less sensitive to parameter 

variations than its nominally optimal counterpart. In 
0 0 

fact, as a a , S ->-0. These results are not philo- 
sophically disturbing in light of the fact that the 
adaptive control was derived to have a reoptimizing 
quality. One would naturally expect a control effort 
which is constantly tending to minimize a given cost index 
to be closer (or equal) to the optimal cost than a system 
operating with a fixed nominally optimal control. 

5.3 Alternative Sensitivity Index 
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To reinforce these ideas geometrically a more in- 
direct attack on the sensitivity problem v/ill be performed. 
Suppose for the moment C (u) is -a monotically increasing 
function of time. One might then be motivated to consider 
a quadratic sensitivity index. Because the original cost 
index is a function of x(t) and u(t), the new sensitivity 
shall also be. Define the sensitivity index N^" as follows: 

N l = iix*ct) -x\t)II 2 + / -xVt), Set) + 

to (5-2) 

+ < , Ruu u*<±> - ul l <±))> dt 

where S(t) and R(t), are positive definite and 

X L <t) = -f 

x^t 0 ) = x 0 (5 " 3) 

i - 

Because N is positive definite, zero is the minimal 
value of N . Consider, for example, the -case where the 
parameter variations from the nominal are sufficiently 
small and known exactly (i.e., a e = a ) such that u (t) = 
u e (t). Then x*(t) = x e (t) and N e = N* = 0 , which implies 
the system is insensitive to such parameter variations. 

If two systems operating with control efforts u 1 (t) and 
u (t) are compared for a given set of admissible parameter 

, A Ir 

variations, and if N J > N over this set of parameter 

. V 

variations, then the system associated with u shall be 
considered to be the "less sensitive" of the two for the 
given parameter variations with respect to N 1 . 

One may compare the sensitivity, in the N 1 sense, 
of the derived adaptive system with the nominally optimal 
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controlled system as follows: 

Consider the dynamical n dimensional system 

uto,a*> , x(t 0 )=rx 0 

with u (t) and u (t) previously defined. 

Condition (i) 

For sufficiently small parameter variations it has 
been established that u e (t) and u (t) may be represented 
as : 


u e (t > = u n ct) + c ; uj c X(t) 

*■ M <-t> < a - a p ) 


(5-4) 


= u. n (-t) *• Q ity (x. t-t) - + 

+ H (t)( a- a"; + oC€ z ) . 


(5-5) 


Combining (5-4) and (5-5) 

u s (t) = u*tt) - o(€ 2 ). 

Consider the following admissible variations 

x(t)* x*(-U + € Sxct), a= a*+€8a 

For the adaptive system (i.e. u(t)- = u e (t)), one achieves 
the variational differential equation 

X t m & + eSxi-t) = Sx(t) 

€***o o € 


S X L"t) = JUm (X*(t) -t- cSxif Oi-cSa, U*(t)-Q(€ 2 X) 

€-^o 55 (5 . 

= 4 C x*tt 8xit> > S a 

Sx{t 0 i = ° 
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where all the partial derivatives exist, are well defined 
and continuous over ie [v T ]- Define the solution of (5.6) 
to be £(t) . 

r T 

£(t> = J ^(t/n^cx^r^a^ucr^S&dT 

to 

where 

• - 

<£ct,T) = •f.CA^ct ),cx* a*a>^Oct,*n 

<£>tT,T)=I. 

Por u(t) = u n (t) 

(iii) 

8 x (-tA = I'twt ( + e 8x<ti , a + e Sa , 

€->a 

3 U*rt) - € &XH) -X n (*t>^ + 

- H(t> ( at €§a- a n > - ocs 2 ) ) 

(5-10) 

= a* <i*c-fc»Sxm +• l a (x*t),a 4 : 

,u*ot>).8a - a*<t) [Q<t) 

Sxet>* H(t)8aj 
Sx<t 0 ) = o 

Define the solution of (5-10) to be 7J(t) . 
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A 

7}Lt)= / <pa,T) 'L(x*cT),a*ucr))Sa aT + 

/•X. 

- J d>c t,T) ^cx^r^a* uV; [(J(r)Sx<T)- 

*•<> 

+ HcriSa] dT 


or equivalently 


Tj (t ) =- £ ct > - L t <S?7 <t)) - L 2 c a ft)) (5-ii ) 

cohere 

L, = / <X X CT),a^ tL*CTl) CjfT) 

* / t . 6 w 

L 2 " f ^ct/n^cx*CT>,a* aV))H(T) 

to 


and and L 2 are bounded linear operators. 

It is now possible to compare the following sensitivity 
indicies: 

N 6 :- the sensitivity index for the adaptive 
(reoptimization) scheme. 

N n : the sensitivity index for the nominally optimal 


controlled scheme. 
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if the reoptimization scheme is to be "less sensi- 
tive" than the nominal scheme for a given set of ad- 
missible parameter varations 

< iv) N e - N°- o. 

This implies 

f Rc-t> 8 u e co> +• 

x 0 

- < S77 ct) jStt) 8 r}(t)> ~ <8 a ”(t), Ret) 8 a n (t)>] dt + 

4-<8fiT>,5cT>8^cT)>-<8j?cT),Rm873|fT)>. 

Theorem 5-1 : For the sensitivity index given by (5-2) 

and Condition (i) satisfied, a sufficient condition that 
N e - N n < 0 is that 


<V) 


2 $ (t) [1,(877 (t))> U(S go] 

II 1,(877 <tv) + L 2 (8a)J| 


£1 


5Ct) 


Proof: 


One notes <5u e (t) = l'i^ ~-)0( s 2 ) — *- 0 , 

and 8a r Vt) = -G(t)8x<tY~ H(t)8d (-from 5-5) 

■fc 

- • J <8 u e (t), Rct)8 uf(t>>dt^ y< 8 lAt), Ret >8 a n (t)>dt „ 

to 

Therefore , for (iv) to hold true, it is sufficient to show 

f { 2< £(t) 3 Sco[ L 2 (Sa)]> + 
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~^L L,(8r} (t))+L z ( 8a)], Set) [ L^S^ct)] + 

+ l 2 (Sa)]>]- dt +2<^CT),Sm[L,(’S77(T))^ 
+ L 2 ( Sa) J > - < [ L, (8r]cn) t-L 2 C 8a), 

, Sen [" L t CSt? m) +L 2 ( 8 a)] > 

^ o. 


A sufficient condition is 

Z < fet), S(t)[ L,f SijfoHljCSa)] > 

— i— ^ ^ i 

II 1; c8rj <■*>)* l 2 ( 8a)H 

for all tc ft o ,Tj . 


Although the structure of the scalar sufficiency 
appears innocent, its computation may be a very complex 
problem. However, the sufficiency condition does rein- 
force some intuitive ideas one may have about ^"(t) and7y(t) 

3 

Consider the following state trajectories in E and their 
intersection with a t = t' hyperplane. 
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Figure 5-1 
STATE TRAJECTORIES 

In the t' hyperplane the following vectors may be 
identified: 



Figure 5-2 
t" HYPERPLAHE 
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The vector 7?( t ') is the solution to the state variational 
equation with u(t) - u n {t), te £t o ,t''j , at t = t". The 
vector ^(t) is the solution to the state variational 
equation with u(t) defined by (5-4), tg at t = t". 

The vector jj5 f (7J( t " ) ) + L 2 (Sct)J is defined by (5-11) at 
t = tl. Inequality (v) (Theorem 5-1) places some restric- 
tions oh the magnitude and orientation of e (t) . For ex- 
ample, some acceptable values of ^(t) are diagramed below. 



Figure 5-3 
ACCEPTABLE £(t") 
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Examples where the sufficiency condition is violated are 
(note: | | Tj (t) || < | j £(t) ||) 



. (MT^+UCgg)) 

2 


<<50 

The angle 0 between f(t> and — (L^ (Tj(t) ) + L 2 (Sot) ) belongs 
to (90°, 270°). 
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Therefore, the first variational argument goes beyond re- 
quiring that the variations in the adaptive control, from 
the true optimal control, be small, or neglected. It re- 
quires that ^(t) be small and have a proper direction. 
Because of the continuity properties associated with the 
variational equations one would expect £ (t) to have a 
direction similar to the direction associated with Tj (t) . 
Therefore, (v) becomes basically a magnitude restriction 
of the form | j £(t) j | < ] |7^(t) | (^ 
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Chapter VI. 

SUMMARY 

6 . 1 Summary 

The problem of sensitivity with respect to parameter 
variations has been sthdied under the philosophical re- 
quirement that optimality be preserved. The device which 
accomplishes this has the property of generating a control 
effort which minimizes 

C (u)= i <x(t>,Tx<t» L(xit),U(t))d£ 

for parameter values belonging to a set of admissible 
parameter variations. The mathematical machinery used to 
develop such a device was a truncated Taylor Series rep- 
resentation of the system's Hamiltonian system of equa- 
tions. Once certain local smoothness and partial deriva- 
tive tests had been satisfied in a neighborhood of the 
nominally optimal- solution, the Maximum (Minimum) 

Principle was applied to the problem. As a result of this 
•action, a set of canonical equations were generated from 
which the matrix Riccati equation evolved as a by-product. 
As a derived result, the adaptive control abstracted from 
these equations was found to be a linear combination of 
the system's states and parameters. 

It was shown that for parameter estimates sufficiently 
close to the system's actual parameters, the cost incurred 
using the adaptive control structure was less than, or at 
worst equal to, that incurred by a system operating under 
a nominally optimal control policy only. That is, under 
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•certain local restrictions the adaptive system’s -control 
effort was optimal, or near optimal, over a set of admissi- 
ble parameter variations. By virtue of this fact, and a 
sensitivity index given by {5-1) , 

1C ia*>-Ccu. r >l 

it was shown that the adaptive system was less sensitive 
to parameter variations than its nominally optimal controlled 
counterpart . 

Numerous techniques were offered to resolve the "curse 
of an adaptive system" which' is parameter estimation. They 
were .basically of two deterministic classes; namely, num- 
erical and gradient techniques. Also, an introduction to 
a particular class of non-deterministic parameter estima- 
tion scheme was' explored with suggestions and computational 
techniques given to satisfy this requirement . 

6.2 Suggestions for Further Research ’ ' . 

The most obvious area requiring additional research is 
that of parameter estimation. The devices and techniques 
which may be developed in this area, because they are after 
all approximation schemes, are limited only by the 
designer's imagination-. 

Numerical experimentation should also prove useful in 
determining a larger class of admissible parameter varia- 
tions than that achieved by analytical means. Another 
numerical study might be to form a linearized approximation 
of a particular given plant to achieve a system of the form 

x •= Aca)x + Ba.. 
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Then mechanize an adaptive controller for this system, and 
perform numerical experiments on it over a set of parameter 
variations, to establish whether or not this technique has 
any merit. 

An interesting analytical study might involve deter- 
mining an optimal selection policy which would define a 
finite set of fixed parameter vectors. It would be re- 
quired that for each parameter vector, say a^, a successful 
reoptimization can be accomplished with an adaptive con- 
troller with an acceptable degree of accuracy. A set of 
fixed dissimilar parameter vectors will be called the 
optimal set of parameter vectors if it satisfies the 
criterion that for an open neighborhood of parameter 

vectors about admitting the previously discussed re- 
optimization, and some given set of ; N of parameter varia- 

. ** *■» ■ „ . 
tion^ about the 'nominal parameter vector U. l is 

minimum, where 



That is, one wishes ’to find the minimal number of open. set 

(i.e., .) which forms an open cover of N., 

1 

Also., controllability and observability were essen- 
tial in constructing the reoptimizing system. A study 
pertaining to the loss of controllability or observability, 
if -indeed it should occur, would prove beneficial. Such 
an analysis would be straightforward . 

The optimization problem considered was of a fixed 
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terminal time class. Extensions of this result into the 
other basic classes of optimal control problems should be 
pursued. Also, the terminal cost index was defined to be 
quadratic. It was assumed to be quadratic to reduce some 
of the computational problems encountered without too 
great a loss of generality. Therefore > it is suggested 
that this terminal cost index might be generalized. 



VII. BIBLIOGRAPHY 


1. Bellman, R. and Kalaba, R. , Ed., Selected Papers on 
Mathematical Trends in Control Theory, Dover, 

pp. 106-123, 1964. 

2. Horowitz, I. M. , "Fundamental Theory on Automatic 
Linear Feedback Control Systems," IRE Transactions 
on Automatic Control , Vol. AC-4, pp 5-19, Dec. 1959. 

3. Horowitz, I. M. , Synthesis of Feedback Systems , 

Academic Press > New York, 1963. 

4. sobral, M. , Jr., "Sensitivity in Optimal Control 

Systems," IEEE Proceedings , Vol. 56, No. 10, pp 1644- 
1652, October, 1968. • ■ 

5. Dorato, P. , "On Sensitivity in Optimal Control Systems," 
IEEE Transactions on Automatic Control , Vol. AC-10, 

pp 495-496, October, 1965. 

6. Pagurek, B., "Sensitivity of the Performance of 
Optimal Control Systems to Plant Parameter Variations," 
IEEE Transactions on Automatic Control, Vol. AC-10, 

pp 178-180, April, 1965. 

7. Kohotonic, P. , and Sannuti, P., "Note on Pagurek- 
Witsenkausen Sensitivity Paradox," IEEE Transactions 
on Automatic ' Control , (To be published) . 

8. Rohrer, R. A., and Sobral, M. , Jr., "Sensitivity 
Considerations in Optimal System Design," IEEE 
Transactions on Automatic Control , Vol. AC-10, 
pp 43-48, January, 1965. 

9. ' Kahne, S. J. , "Low Sensitivity Design of Optimal 

Linear Control Systems , " Proceedings of the 4th 
Annual Allerton Conference on Circuit and System 
Theory , University of Illinois, Urbana, Oct., 1965, 
pp 225-229 . 

10. D'Angelo, H. D. , Moe, M. L., and Hendricks, "Trajectory 
Sensitivity of an Optimal Control System," Proceedings 
of the 4th Annual Allerton Conference on Circuit and 
System Theory , University of Illinois, Urbana, 1965, 
pp 489-498. ' 



142 


11. Bradt, A., "The Design of Optimal Controllers to 
Minimize a Performance Index Containing Sensitivity 
Functions," Ph.D. Dissertation , Department of 
Electrical Engineering, University of Denver, 

October, 1967. 

12. Athens and Folb, Optimal' Control , McGraw-Hill, 1966. 

13. Hartman, P., Ordinary Differential Equations , 

John Wiley & Sons, Chapter V, 1964. 

14. Fulks , W. , Advanced Calculus , Wiley, P 230, 1961. 

15. Milkhin, S. G. , The Problem of the . Minimum of a 
Quadradic Functional , Holden Day, 1965. 

16. Simmons, G., Topology and Modern Analysis ,■ McGraw- 
• Hill, pp 126-128, 1963.- 

17. Tuel, W. , "An Improved Algorithm for the Solution 
of Discrete Regulation Problems,"' IEEE Transactions 
on Automatic . Control , Vol. AC-12, pp 522-527, 

October, 1967. 

18. Friedland, B., "On Solutions of the Ricatti Equation 
in Optimization Problems," IEEE Transactions on 
Automatic Control , Vol. AC-1], pp, 303-304, June, 1967. 

19. Bass, R. , "Machine Solution of High-Order Matrix 
Riccati Equations," Douglas Missile and Space 
Division Publication , 1967. 

20. Pontryagin, Boltyanski, Jamkrelidze, Mischenko, 
Mathematical Theory of Optimal Control Processes , 
Interscience Pub. , p. 75, 1962. 

21. Kelly, L. , Handbook of Numerical Methods and Applica - 
' tions , Addison Wesley, Ch. 4 and Ch. 7, 1967. 

22. Mortensen, R. E., "A Note on Polar Decomposition and 
the Generalized Inverse of Arbitrary Matrix," Set of 
notes under Grant No. AF-AFOSR-139-64 . 

23. Penrose, R. , "Generalized Inverse of Matrices," 
Cambridge Phil. Society , NO. 55, pp 406-413, 1955. 

24. Rudin, W. , Principles of Mathematical Analysis, 
McGraw-Hill, p 192, 1964. 

Luisternich and Sobolev,' Elements of Functional 
Analysis , Ungar, pp 182-187 , 1961. 


25 . 



143 


26. Halmos, P., Measure Theory , Van Nostrand., p 85, 1959 . 

27. Margolis, M. , Leohdes , C. , "A Parameter Tracking Servo 
for Adaptive Systems," ASTIA Document, Project No. 
9783, pp 5-6, 1959. 

2 8 . Lee , E . , Markus , L . , Foundations of Optimal Control 
Theory , John Wiley and Sons, Appendix A, 1967. 

29. ' Valentine, F. , "The Problem of Lagrange with 

Differential Inequalities with Added Side Conditions," 
Ph.D. Thesis , Department of Mathematics, University 
of Chicago, 1937. 

30. Backman, G., and Narici, L.‘, Functional Analysis, 
Academic Press, pp 209-211, 1968. 

31. Aoki, M. , Optimization of Stochastic Systems , Academic 
Press, Chapter 2, 1967. 

32. Hermes, H., Discontinuous Vector Fields and Feedback 
Control , Reprint from Differential Equations and 
Dynamical Systems, Academic Press, pp 155-165, 1967. 



